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ABSTRACT 


Three  discrete  reliability  growth  models  using  fractional  failure  reduction,  referred 
to  as  failure  discounting,  were  developed  to  estimate  changing  system  reliability.  Each 
of  the  models  is  designed  for  use  when  testing  is  performed  until  a  fixed  number  of 
failures  have  been  observed  and  attribute  data,  success  or  failure,  is  available  for  each 
trial.  The  first  reliability  growth  model  applies  failure  discounting  to  the  maximum 
likelihood  estimate  for  a  proportion.  The  second  and  third  models  use  a  modification  of 
an  exponential  reliability  estimate  employing  linear  regression  and  a  weighted  average 
technique  respectively  along  with  failure  discounting  to  track  changing  reliability.  Two 
failure  discounting  methods  were  used  with  each  reliability  growth  model.  The  lirsf 
method  reduces  past  failures  by  a  fixed  fraction  at  a  fixed  interval.  The  second  method 
uses  the  upper  confidence  bound  for  the  reoccurrence  of  each  failure  cause  as  the 
discounted  failure  value.  The  performance  of  the  reliability  growth  models  with 
varying  reliability  growth  patterns  w’as  evaluated  with  a  .Monte-Carlo  simulation. 
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THESIS  DISCLAIMER 


The  reader  is  cautioned  that  computer  programs  developed  in  this  research  may 
not  have  been  exercised  for  all  cases  of  interest.  While  every  effort  has  been  made, 
wthm  the  time  available,  to  ensure  that  the  programs  are  free  of  computational  and 
logic  errors,  they  cannot  be  considered  validated.  .Any  application  of  these  programs 
without  additional  venfication  is  at  the  risk  of  the  user. 
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I.  INTRODUCTION 

Accurately  determining  the  reliability  of  a  newly  designed  piece  of  hardware  is  an 
important  part  of  material  acquisition  in  both  the  military  and  civilian  sectors. 
Significant  amounts  of  both  time  and  money  are  spent  testing  new  systems  to 
determine  if  they  meet  specified  reliability  requirements.  Since  most  organizations  are 
under  constant  pressure  to  conserve  both  time  and  funds,  it  is  important  to  develop 
methods  of  estimating  system  reliabihty  at  minimum  cost.  When  conventional 
estimators  for  system  reliability  are  used,  verifying  high  reliability  requires  a  great  deal 
of  test  data  costing  both  time  and  money.  This  problem  is  compounded  by 
technological  advances  which  produce  more  complex  and  highly  reliable  hardware  at 
an  ever  increasing  cost.  Developing  new  and  more  accurate  methods  of  estimating 
reliability  that  reduce  required  testing  could  result  in  significant  savings  of  resources. 

Early  versions  of  newly  designed  equipment  often  exhibit  low  reliability.  As 
engineering  designs  are  tested  and  improved  the  system  reliability  usually  grows 
rapidly.  This  process  continues  throughout  development  phases  and  into  field  use.  By 
the  time  a  product  design  reaches  its  fmal  configuration,  a  large  amount  of  test  data  is 
usually  available  from  tests  performed  on  previous  configurations,  subcomponents  and 
parts.  Since  conventional  estimators  assume  a  constant  reliability,  it  is  necessary  to 
wait  until  the  finished  piece  of  hardware  is  available  to  begin  reliability  verification. 
This  practice  is  not  an  efficient  use  of  constrained  resources  because  results  from 
previous  testing  are  discarded,  [f  all  the  data  available  could  be  used  to  estimate  final 
configuration  reliability,  the  lead  time  and  cost  of  the  finished  product  could  be 
reduced.  Reliability  growth  models  serve  this  purpose.  They  use  data  from  testing  at 
all  phases  of  development  to  establish  a  reliability  growth  pattern.  This  growth  pattern 
can  then  be  used  to  provide  current  estimates  of  system  reliability  with  less  testing  than 
standard  testing  programs. 

Reliability  gro\nh  models  can  be  classified  into  two  categories,  discrete  and 
continuous  models.  This  paper  will  address  discrete  reliability  growih  models  that 
employ  failure  discounting.  These  growth  models  are  used  to  determine  reliability  for 
systems  where  test  results  can  only  be  staled  in  terms  of  success  or  failure,  le;  attribute 
data.  Failure  discountmg  allows  the  user  to  fractionally  discount  past  failures  when 
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resultant  corrections  to  equipment  are  verified  through  subsequent  testing,  as  actual 
corrections  to  the  cause  of  failure.  The  use  of  failure  discounting  makes  it  possible  for 
previously  linear  estimators  to  track  a  non-linear  growth  pattern. 

The  objective  of  this  paper  is  to  develop  and  evaluate  the  performance  of  discrete 
reliability  growth  models  using  failure  discounting.  Three  different  reliability  growth 
models  are  proposed  and  evaluated  with  two  different  methods  for  discounting  past 
failures.  Monte-Carlo  simulation  is  used  to  evaluate  each  of  the  three  growth  models 
under  both  discounting  methods.  The  simulation  uses  actual  reliabilities  to  generate  a 
random  growth  pattern.  From  this  growth  pattern,  test  data  is  then  generated  for  the 
known  system  reliability  at  each  testing  phase.  Each  of  the  reliability  growth  models 
can  then  be  applied  to  the  test  data  producing  an  estimated  growth  pattern.  The 
performance  of  each  reliability  growth  model  is  evaluated  by  comparing  the  estimated 
growth  pattern  w’ith  the  actual  reliability  in  each  phase. 

The  following  chapter  will  address  each  of  the  three  reliability  growth  models 
used  and  the  two  methods  of  failure  discounting  employed.  Chapter  III  contains  a 
discussion  of  the  computer  simulation  used  to  evaluate  each  of  the  models.  Chapter  IV 
presents  the  results  of  the  evaluation  and  Chapter  V  summarizes  the  study  stating 
conclusions  and  recomendations  for  future  study. 


11.  DISCRETE  RELIABILITY  GROWTH  MODELS 


A.  BACKGROUND 

Three  different  reliability  growth  models  and  two  different  techniques  for 
fractionally  discounting  past  failures  are  discussed  in  this  chapter.  The  reliability 
growth  models  can  be  divided  into  two  categories.  .Models  using  a  modification  of  the 
maximum  likelihood  estimate,  MLE,  for  a  proportion,  referred  to  as  the  .MLE  With 
Discounting  .Model,  a  model  referred  to  as  the  Weighted  .Average  Model  and  an 
Exponential  Regression  Model.  All  of  the  models  may  be  applied  with  or  without 
failure  discounting  and  are  independent  of  the  discounting  method. 

Two  different  methods  for  fractionally  discounting  past  failures  are  also 
discussed.  The  first  method  is  referred  to  as  the  Standard  or  Straight  Percent 
Discounting  .Method.  The  second  was  proposed  by  David  K.  Lloyd  and  will  be 
referred  to  as  the  Lloyd  Method.  Before  the  specific  details  of  each  growth  model  and 
discounting  method  are  discussed,  several  terms  need  clarification. 

Reliability,  R,  as  referred  to  in  this  study  is  defined  as  the  probability  of  a  system 
successfully  completing  a  single  trial  or  operation  cycle.  Equation  2.\  .  R  will  be  used 
to  denote  an  estimate  of  the  actual  reliability  R. 

R  =  Pr  (  SINGLE  TRIAL  SUCCESS  )  (eqn  2.1) 

,  Discrete  reliability  growth  models  are  normally  used  for  trials  which  can  only  be 
evaluated  in  terms  of  a  binary  variable,  success  or  failure,  i.e.  attribute  data.  Exactly 
what  constitutes  a  trial  is  normally  defined  by  the  test  agency  but  could  include,  for 
example,  a  single  rocket  engine  ignition  attempt  or  a  single  test  firing  of  a  missile  up  to 
and  including  target  destruction.  To  use  the  reliability  growth  models  discussed  here, 
trials  must  be  defined  such  that  they  are  discrete  and  can  be  evaluated  as  a  success  or  a 
failure. 

.A  testing  phase  is  defined  as  a  collection  of  trials,  one  or  more,  for  which  the 
configuration  of  the  system  being  tested  is  unchanged.  This  implies  that  the  actual 
system  reliability  remains  constant  throughout  a  single  phase.  If  a  test-fix-test 
methodology  is  employed,  design  changes  or  "fixes"  are  implemented  after  each  failure. 


In  this  case,  a  phase  will  consist  of  one  failed  trial  and  all  the  successful  trials  leading 
up  to  the  single  failure.  If  on  the  other  hand  a  test-find-test  methodology'  is  used,  a 
single  phase  may  include  numerous  failures.  Under  both  methodologies,  a  phase  is  the 
collection  of  all  trials  between  configuration  changes.  [Ref  1] 

Because  the  underlying  reliability  of  the  system  may  be  different  from  phase  to 
phase,  a  new  estimate  of  reliability  must  be  computed  for  each  and  every  phase.  .As 
such,  .denoting  system  reliability  and  denoting  the  estimate  of  system  reliability- 
in  phase  k,  will  be  used  throughout  the  paper. 

.Annotating  the  cause  of  each  failure  is  critical  to  all  discounting  methods 
considered  in  this  study.  How  the  failure  cause  is  defined,  is  dependent  upon  the 
judgement  of  the  model's  user  and  the  type  of  system  to  which  the  model  is  applied.  If 
the  model  is  used  to  determine  the  reliability  of  a  single  component,  failure  cause  refers 
to  the  reason  for  failure  such  as  a  broken  cam-shaft  or  fuel  stoppage.  If  the  model  is 
applied  to  determine  the  reliability  of  a  system  of  components,  the  failure  cause  may 
refer  to  the  failure  of  a  sub-component  or  the  specific  reason  for  failure.  To  apply  the 
discounting  methods  discussed  in  the  paper,  the  model  user  must  determine  if  two 
observ'ed  failures  are  due  to  the  same  cause.  It  will  not  be  necessary,  however,  to 
enumerate  all  possible  failure  causes.  The  level  of  detail  at  which  failure  causes  are 
defined  is  up  to  the  user  and  should  be  determined  before  any  testing  is  begun.  The 
discounting  methods  studied  will  be  discussed  in  the  next  section. 

B.  DISCOUNTING  METHODOLOGY 

Weaknesses  in  a  system  design  become  known  as  they  cause  failures  during 
testing.  .As  weaknesses  are  identified  and  corrective  "fixes"  are  implemented  the 
probability  of  a  repeat  failure  due  to  the  corrected  cause  should  be  reduced  and  system 
reliability  improved.  This  concept  forms  the  basis  for  failure  discounting.  Once  a 
system  weakness  has  been  corrected  and  the  improvement  validated  through  further 
testing,  fractionally  discounting  the  past  failure  is  a  method  of  reflecting  the  improved 
system  reliability  in  the  previously  collected  data.  W'hen  using  standard  reliability 
estimators  such  as  the  maximum  likelihood  estimate.  Equation  2.2 


R  =  - 


Number  of  Successful  Trials 
Total  Number  of  Trials 


(eqn  2.2) 


only  the  data  from  the  phase  of  interest  can  be  used  to  determine  system  reliability 
because  system  reliability  is  different  from  phase  to  phase.  If  the  proper  discounting 
methodology  can  be  developed,  failure  data  from  all  past  phases  could  be  discounted 
and  made  compatible  with  the  data  from  the  current  phase.  Increasing  the  amount  of 
data  available  for  computing  the  estimate  of  current  system  reliability  will  increase  the 
accuracy  of  the  estimate  and  reduce  the  amount  of  testing  necessary  to  verify  system 
reliability. 

Of  equal  importance  to  a  method  for  discounting  past  failures  is  a  penalty  system 
that  will  restore  all  or  some  portion  of  a  previously  discounted  failure  if  its  failure 
cause  should  reoccur.  If  a  previously  implemented  “fix"  proves  not  to  have  corrected  a 
design  w'eakness,  the  discounts  applied  to  its  failures  will  diminished  -  erased  reversing 
the  effect  of  discounting.  A  properly  designed  penalty  factor  will  make  the  discounting 
method  self-correcting. 

In  order  for  a  discounting  method  to  be  useful,  it  must  be  flexible  enough  to 
handle  many  varied  reliability  growth  patterns.  To  achieve  this  flexibility,  discounting 
methods  with  one  or  more  input  parameters  are  proposed.  Regardless  which 
discounting  method  is  used,  experience  and  good  engineering  judgement  will  be 
necessary  to  choose  input  parameters  that  are  compatible  with  both  the  reliability 
Estimator  used  and  the  proposed  test  plan.  If  improper  parameters  are  applied,  the 
model  may  consistently  over  or  underestimate  system  reliability.  The  following 
sections  describe  the  specific  steps  for  applying  two  failure  discounting  methods 
including  examples  of  their  use. 

1.  Straight  Percent  Discounting  Method 

The  Straight  Percent  Discounting  Method  removes  a  fixed  fraction  of  a  failure 
each  time  a  predetermined  number  of  trials  are  observed  without  reoccurrence  of  the 
failure  cause.  Before  a  failure  is  discounted,  the  design  "fix"  correcting  the  failure  cause 
should  be  verified  by  further  testing.  This  concept  forms  the  basis  for  the  discounting 
method.  The  fraction  of  a  failure  removed,  F,  is  referred  to  as  the  discount  fraction. 
.After  the  initial  occurrence  of  a  particular  failure  cause,  any  trial  in  which  the  tauurc 
cause  is  not  repeated  is  considered  a  success.  The  number  of  successful  trials  between 
application  of  the  discount  fraction  is  referred  to  as  the  discount  interval.  N.  Both  N 
and  F  are  input  parameters  specified  by  the  u.ser.  They  remain  constant  throughout 
failure  discounting  computations.  To  use  the  Straight  Percent  Discounting  Method,  it 
is  necessaiy  to  record  the  number  of  successful  trials  since  the  last  occurrence  of  each 


observed  failure  cause,  referred  to  as  M.  Failure  discounting  is  performed  at  the  end  of 
each  testing  phase  using  the  values  for  N,  F  and  M. 

The  mathematical  representation  of  the  straight  percent  discounting  method  is 
shown  in  Equation  2.3 

ADJUSTED  FAILURE  =  (1  -  (eqn  2.3) 

F  =  Fraction  of  a  Failure  to  be  Removed 

M  =  Number  of  Successful  Trials  Since  Last  Failure  for  the  Failure  Cause 
N  =  The  Discount  Interv'al  Expressed  in  the  Number  of  Successful  Trials 

INT  is  a  function  which  returns  the  truncated  integer  portion  of  its  argument.  M  is  set 
to  zero  at  each  occurrence  or  reoccurrence  of  a  particular  failure  cause.  This  causes  the 
value  for  adjusted  failure  to  be  set  at  1.0  erasing  any  previously  applied  discounting. 
This  also  functions  as  a  penalty  factor  allowing  the  model  to  correct  itself  should  past 
discounting  prove  unwarranted.  By  altering  the  choice  of  N  and  F  the  amount  of 
discounting  applied  can  be  varied  from  none  to  total  loss  of  a  failure.  This  allows  the 
method  to  be  applied  in  situations  ver>'ing  from  little  or  no  reliability  growth  to  rapid 
reliability  growth. 

An  example  will  help  clarify  the  use  of  the  method.  Consider  the  test  results 
included  in  Table  1.  The  data  displayed  in  the  table  is  from  a  test-fi.x-test  program  in 
time  order  sequence.  .A  design  fix  was  applied  to  the  equipment  after  each  failure.  .As 
such,  the  actual  system  reliability  does  not  remain  constant.  The  outcome  of  each  trial 
was  generated  from  the  geometnc  distribution  using  the  actual  reliability  listed  in  the 
table.  The  cause  for  each  failure  was  generated  using  a  stochastic  process  and  has 
been  coded  using  letters.  No  more  information  concerning  failure  cause  is  required 
because  the  data  are  used  to  determine  if  the  current  failure  cause  has  occurred 
previously. 

To  apply  Straight  Percent  Discounting  the  two  input  parameters  must  be 
determined.  For  the  example 
N  =  3 
F  =  .25 

Will  be  used.  The  Straight  Percent  Discounting  Method  is  applied  sequentially,  failure 
by  failure,  recompur-ng  M,  the  number  of  successful  tnals  since  last  failure,  for  each 
failure  cause  as  the  next  data  entn.’  is  included.  .At  the  end  of  each  testing  phase,  the 


TABLE  1 

SAMPLE  TEST  RESULTS 


TRIAL 

FAILURE 

FAILURE  CAUSE 

NUMBER 

NUMBER 

A 

B 

c 

D 

RELIABILITY 

1 

S 

S 

s 

s 

.358 

2 

1 

F 

S 

s 

s 

.358 

S 

F 

s 

s 

.412 

4 

S 

S 

F 

s 

.518 

5 

4 

S 

S 

s 

1* 

.596 

6 

S 

S 

s 

s 

.627 

5 

S 

S 

F 

s 

.62^ 

8 

S 

s 

s 

s 

.721 

9 

S 

s 

s 

s 

.721 

10 

6 

s 

F 

s 

s 

.721 

U 

s 

s 

s 

s 

.780 

12 

s 

s 

s 

s 

.780 

13 

s 

s 

s 

s 

.780 

14 

S 

s 

s 

s 

.780 

15 

S 

s 

s 

s 

.780 

16 

s 

s 

s 

s 

.780 

17 

s 

s 

s 

s 

.780 

18 

/ 

F 

s 

s 

s 

."SO 

S  :  DENOTES  SUCCESS 
F  ;  DENOTES  FAILURE 

value  of  \I,  along  with  the  input  parameters  F  and  N.  is  used  in  Equation  2.3  to 
determine  the  adjusted  failure  value.  This  procedure  is  applied  to  the  data  in  Table  1. 

When  the  si.xth  failure  is  included,  all  failures  due  to  Causes  A,  C  or  D  have 
values  for  M  greater  than  or  equal  to  three  causing  the  discount  fraction  to  be  applied. 
Computation  of  the  adjusted  failure  values  for  Cause  A  is  illustrated  in  the  following 
calculations. 


ADJUSTED  FAILURE  i  - 

=  ( .75  f 
=  .5625 

Failure  two  and  failure  six  were  both  due  to  Cause  B  which  resets  their  corresponding 
M  values  to  zero  and  their  adjusted  failure  value  to  unity.  The  results  for  all  failures  to 
date  are  shown  below. 


FAILURE 

FAILURE 

ADJUSTED 

NUMBER 

M 

CAUSE 

FAILURE 

1 

8 

CAUSE  A 

.5625 

2 

0 

CAUSE  B 

1.0000 

3 

3 

CAUSE  C 

.7500 

4 

5 

CAUSE  D 

.7500 

5 

CAUSE  C 

.7500 

6 

0 

CAUSE  B 

1.0000 

Finally,  failure  seven  is  included  and  the  failure  discounting  computations  are 
completed.  The  number  of  trials,  eight,  causes  further  discounting  for  all  previous 
failures  except  for  the  Enst  failure  which  is  also  due  to  Cause  A.  The  fmal  results  of 
applying  the  straight  percent  discount  method  to  the  data  in  Table  1  appears  in  Table 
2.  The  values  in  the  adjusted  failure  column  are  the  current  values  of  each  fractional 
failure  after  trial  number  eighteen  has  been  included.  For  example,  the  tw^o  adjusted 
failure  values  due  to  Cause  B  have  recorded  eight  successful!  trials  since  the  last  failure 
due  to  Cause  B  and  INT  (8  3)  =  2.  Therefore,  the  adjusted  failure  value  equals 
(.75)^  =  .5625.  The  remaining  adjusted  failures  are  computed  in  the  same  marmer 
yielding  the  fractional  failure  values  after  eighteen  trials  and  seven  failures. 

2.  Lloyd  Failure  Discounting  Method 

The  underlying  premise  of  the  Lloyd  Discounting  Method  states  that  the 
fraction  of  a  failure  removed  by  the  discounting  method  should  not  be  arbitrarily 
chosen.  Seme  statistical  basis  should  be  used  to  determine  how  much,  if  any,  a  past 
failure  should  be  discounted.  Lloyd  proposes  using  the  upper  confidence  limit  for  the 
probability  that  a  failure  cause  will  reoccur  as  the  discounted  value  of  its  corresponding 
failure.  This  technique  allows  the  user  to  control  the  amount  failures  are  discounted  by 


1" 


TABLE  2 

STR.AIGHT  PERCENT  DISCOUNTING 
OF  EX.AMPLE  DATA  <  TABLE  1  ) 


TRIAL  FAILURE  FAILURE  CAUSE  ADJUSTED 


NUMBER 

NU.MBER 

A 

B 

c 

D 

FAILURE 

I 

S 

S 

s 

s 

2 

1 

F 

S 

s 

s 

1.0000 

3 

2 

S 

F 

s 

s 

.5625 

4 

3 

S 

S 

F 

s 

.4219 

5 

4 

S 

S 

S 

F 

.3164 

6 

S 

S 

S 

s 

7 

5 

S 

S 

F 

S 

.4219 

8 

S 

S 

S 

S 

9 

S 

s 

S 

s 

10 

6 

S 

F- 

S 

s 

.5625 

11 

s 

S 

S 

s 

12 

s 

s 

S 

s 

13 

S 

s 

S 

s 

14 

S 

s 

s 

s 

15 

S 

s 

s 

s 

16 

s 

s 

s 

s 

17 

S 

S 

s 

S 

18 

7 

F 

s 

S 

s 

1. 0000 

S  ;  DENOTES  SUCCESS 
F  .  DENOTES  FAILURE 


specifying  the  leve’  of  confidence  for  the  confidence  bound  As  such,  a  single 
parameter,  the  confidence  level  y,  is  required  to  perform  failure  discounting.  [Ref. 


input 


IS 


The  discounting  equation  is  denved  from  the  upper  confidence  limit  for  the 
probability  of  failure  due  to  a  panicular  failure  cause.  This  confidence  interval  is  based 
on  the  number  of  successful  trials  recorded  since  implementation  of  the  "fix"  for  the 
failure  cause.  Assuming  the  "fix"  is  implemented  immediately  after  failure,  this  '.nil  be 
equivalent  to  the  number  of  successful  tnals  since  last  failure,  previously  defined  as  VI. 

Q  *  PROBABILITY  OF  FAILURE  IN  A  SINGLE  TRIAL 
=  I  -  R 

Y  =  FR.\CTIONAL  CONFIDENCE  LEVEL 

.M  =  NU.VIBER  OF  SUCCESSFUL  TRIALS  SINCE  LAST  FAILURE 
=  LOWER  Y%  CONFIDENCE  LIMIT  FOR  RELIABILITY 

Su-I)  =  ('  -  v)‘ 

•/  =  Pr  [  R  a  1 
=  Pr[  l-R  :S  ^"5l(y)1 
=  Pr[Q  tS  1-Sl(y)1 

•••  UPPER  CONFIDENCE  LIMIT  for  Q  =  1  -  (  I  ~  Y  ^  (eqn  2.4) 

ADJUSTED  FAILURE  =  1  -  (  I  -  Y  ^  for  M  >  0  (eqn  2.5) 

=  1.0  for  M  =  0 

The  failure  discounting  equation  is  derived  in  Equations  2.4  and  2.5.  Because  the 
derived  equation  for  adjusted  failures  is  not  defined  for  M  equal  zero,  the  adjusted 
failure  value  is  set  equal  to  one  where  .M  equals  zero.  This  is  the  logical  choice  since  no 
discounting  should  be  applied  to  a  failure  cause  that  is  the  current  cause  of  system 
failure.  This  also  has  the  advantage  of  providing  a  penalty  factor  that  will  restore 
fractional  failures  to  unity  if  a  failure  cause  reoccurs. 

To  apply  the  Lloyd  discounting  method,  the  confidence  level,  y.  rnust  be 
specified.  The  method  is  then  applied  sequentially,  failure  by  failure,  recomputing  M, 
the  number  of  successful  trials  since  last  failure,  for  each  observed  failure  cause  as  the 
data  from  each  trial  is  included.  .At  each  step,  M  is  used  in  Equation  2.5  to  compute  a 
new  adjusted  failure  value  for  each  observed  failure. 


An  example  will  help  clarify  the  use  of  the  Lloyd  method.  Applying  the 
method  to  the  sample  data  in  Table  I  with  y  equal  to  .99  yields  the  following  results 
after  the  second  failure  is  included. 

FAILURE  FAILLRE  ADJUSTED 

N'UMBER  M  CAUSE  FAILURE 

1  1  CAUSE  A  .9900 

2  0  CAUSE  B  1.0000 

Including  the  data  for  the  third  failure,  due  to  failure  Cause  C,  both  Cause  A  and  B  are 
further  discounted.  The  adjusted  failure  value  for  Cause  A  is  computed  below. 

M  =  2 

ADJUSTED  FAILURE  =  1 

=  1 
=  1 
=  .9 

The  complete  results  are  shown  below. 

FAILURE  FAILURE  ADJUSTED 

N’UMBER  M  CAUSE  FAILURE 

1  2  CAUSE  A  .9000 

2  I  CAUSE  B  .9900 

3  0  CAUSE  C  l.OOOO 

The  process  continues  sequentially,  adding  the  next  failure,  recomputing  M,  and 
computing  the  adjusted  failure  values  for  each  previous  failure.  The  final  results  after 
ail  seven  faiiurcs  arc  .nciuded  is  shown  in  Table  3.  Note,  as  a  failure  cause  reoccurs  its 
corresponding  value  for  M  is  set  to  zero  forcing  the  adjusted  failure  value  to  unity. 

This  concludes  the  description  of  the  two  discounting  methods  covered  in  the 
study.  The  following  section  will  describe  the  three  discrete  reliability  growth  models. 


1  /O 

-  (  I  -  .99 

-  J  .01 

-  .1 


TABLE  3 

LLOYD  DISCOUNTING 
OF  EXAMPLE  DATA  (  TABLE  1  ) 

TRIAL  FAILURE  FAILURE  CAUSE  ADJUSTED 


NUMBER 

NUMBER 

A 

B 

C  D 

FAILURE 

1 

S 

S 

S 

S 

2 

1 

F 

s 

S 

S 

1 .0000 

3 

2 

S 

F 

S 

S 

.4^  /  / 

4 

3 

S 

S 

F 

S 

.3421 

5 

4 

S 

S 

S 

F 

.2983 

6 

S 

S 

s 

S 

5 

S 

S 

F 

S 

.3421 

8 

S 

S 

S 

S 

9 

s 

s 

s 

s 

10 

6 

s 

F 

s 

s 

.4377 

11 

s 

S 

s 

s 

12 

s 

S 

s 

s 

13 

s 

S 

s 

s 

14 

s 

S 

s 

s 

15 

s 

S 

s 

s 

16 

s 

S 

s 

s 

r 

S 

S 

s 

s 

18 

F 

s 

s 

s 

l.(Jt)Ot) 
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C.  RELIABILITY  GROWTH  MODELS  DESCRIPTION 

Of  part'.cular  interest  in  this  stuu>  are  test  plans  which  terminate  each  test  phase 


uith  '•.stern  laiiare  Such  test  plans  are  kno-.sn  as  testing  ta  a  fixed  number  of  failures. 


Only  one  of  the  three  reliahilitv  grovsth  models  proposed  can  be  used  for  test  scenarios 


which  terminate  testing  after  a  fixed  number  of  trials.  Because  of  this  limitation,  only 
test  plans  which  test  to  a  fixed  number  of  failures  are  considered  in  this  study. 

Three  discrete  reliability  growth  models  are  presented  in  this  section,  each  of 
which  uses  a  different  technique  to  track  changing  system  reliability.  The  first  model. 
Maximum  Likelihood  Estimate  With  Discounting,  is  derived  from  the  eonventiona. 
maximum  likelihood  estimate  of  reliability.  Failure  discounting  gives  the  model  the 
ability  to  track  changing  reliability.  Two  additional  models  derived  from.  ar. 
exponential  single  phase  reliability  estimate  referred  to  as  the  Weighted  .\verage  M  laei 
and  the  Exponential  Regression  Model  are  also  discussed.  The  MLE  V.  ;:r. 
Discounting  model  will  be  presented  in  the  next  section. 

1.  Maximum  Likelihood  Estimate  With  Failure  Discounting 

The  most  commonly  used  estimate  of  system  reliability  is  the  m.aMmum. 
likelihood  estimate  Equation  2.2  restated  below. 

Number  of  Successful  Trails 

R  =  - - —  leqn  2.2  ' 

Total  Num.ber  ot  Trials 

Where  the  nutnber  of  successful  trials  equals  the  total  number  of  tnals  minus  'he 
number  of  observed  failures.  Because  all  testing  considered  is  conducted  to  a  fixed 

NO.  SUCCESSFUL  TRI.ALS  =  ^  (  T  -  I  )  (eqn  2.6) 

i=  1  ' 


num.ber  of  ."ailures.  the  number  of  successful  trials  simplifies  to  Equation  2.0.  ^^’here 
equals  the  number  of  trials  following  the  r-  1)  '''■  failure  .nciading  tiie  .  ■'  laiiure  inu 
k  equals  the  total  num.ber  of  failures  in  the  test  program  'o  date. 

Two  problem.s  anse  when  using  Equation  2.2  to  estimate  reliability  growth. 
First,  the  esti.matc  takes  a  large  number  of  tnals  to  accurately  esti.mate  svstem 
reliabiht".  Se..::nd.  fite  .'Sti.m.ator  requires  i  ..onstant  reliability,  R..  .or  eai.ii  ri..!. 
Because  the  reiiabilir.  ot  eacii  phase.  R|^.  may  not  ioe  constant  wuh  k,  or.i;.'  the  test 
data  I'rcm  the  phase  of  interest  may  be  used  to  estimate  reiiabiiity.  Failure  discounting 
IS  introduced  into  the  model  m  an  attempt  to  adjust  the  underlying  rehabiiity  for  data 
from  previous  phases,  m.aking  it  compatible  with  the  data  from  the  current  piaase.  This 
increases  the  amount  of  data  available  for  estimating  reliability  at  each  phase. 


improving  the  accuracy  of  the  estimate.  Choosing  the  correct  discounting  method  with 
the  correct  parameters  is  critical  to  this  model  because  the  previous  test  data  must  be 
properly  adjusted  to  reflect  an  underlying  reliability  that  is  compatible  with  the  current 
phase.  This  problem  is  difficult  to  solve  because  the  actual  reliability  in  each  phase  is 
unknown. 

Introducing  failure  discounting  creates  a  computation  problem  because  the 
number  of  failures  is  no  longer  an  integer  value.  This  problem  can  be  resolved  in  one 
of  two  ways.  The  failure  can  remain  a  fractional  value  or  the  failure  value  can  be  reset 
to  unity  by  adjusting  the  observed  number  of  trials  up  to  and  including  failure.  Which 
ever  method  is  chosen,  the  ratio  of  failures  to  total  trials  must  remain  constant  or  the 
estimate  of  reliability  will  be  altered.  Returning  the  failure  value  to  unity  and  adjusting 
the  number  of  trials  is  the  method  chosen  to  resolve  this  problem.  This  solution  allows 
the  data  to  retain  its  original  structure  of  the  number  of  trials  up  to  and  including 
failure. 

.Adjusting  fractional  failures  and  the  number  of  trials  up  to  and  including 
failure  is  accomplished  by  dividing  both  values  by  the  fractional  failure  value.  This 
returns  the  number  of  failures  to  unity  and  adjusts  the  number  of  trials  such  that  the 
ratio  between  the  number  of  failures  and  the  number  of  trials  remains  constant.  The 
adjusted  number  of  trials  may  not  be  an  integer  value.  This  does  not  present  a  problem 
for  computing  the  MLE.  In  cases  where  the  total  number  of  trials  must  also  be  an 
integer  value,  the  adjusted  number  of  trials  is  rounded  to  the  nearest  integer.  This  will 
have  a  minimal  effect  on  the  ratio  of  failures  to  trials. 

For  example  the  following  data  is  the  result  of  applying  a  failure  discounting 
method  to  observed  test  data. 

FR.ACTIONAL  FAILURE  =  .5 
NUMBER  OF  TRIALS  =  6 

Dividing  both  values  by  .5  adjusts  the  fractional  failure  to  unity  and  yields  an  adjusted 
failure  value. 

FAILURE  VALUE  =  1.0 
ADJUSTED  NUMBER  OF  TRIALS  =  12 
The  ratio  of  failures  to  trials  remains  constant. 

.5/6  =  1T2  =  .0833 
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In  general.  Equation  2.7  is  used  to  adjust  the  number  of  trials  when  the  adjusted  failure 


ADJUSTED  #  TRIALS  = 


OBSERVED  #  of  TRIALS 
ADJUSTED  FAILURE 


(eqn  2.7) 


value  is  returned  to  unity. 

The  reliability  growth  model  is  applied  to  the  data  sequentially,  phase  by 
phase.  After  the  test  data  from  a  phase  is  recorded,  the  discounting  routine  is  applied 
to  the  data  and  an  adjusted  failure  value  is  computed  for  each  observed  failure.  The 
adjusted  number  of  trials  for  each  observed  failure  is  computed  using  Equation  2.'*. 


4  = 


(X  ADJUSTED  TRIALS)  -  J 
X  ADJUSTED  TRIALS 


(eqn  2.8) 


J  =  TOTAL  .number  OF  FAILURES  IN  THE  TEST  PROGRAM  TO  DATE 


Finally,  the  estimate  of  system  reliability  in  phase  k  is  computed  using  Equation  2.8. 
This  procedure  is  repeated  after  each  testing  phase  is  completed  yielding  an  estimate  of 
system  reliability  for  each  phase.  These  estimates  can  then  be  used  to  plot  the 
estimated  reliability  growth  pattern. 

To  illustrate  the  use  of  the  model,  the  discovited  failure  values  from  Table  2 
are  used  to  compute  an  estimate  of  system  reliability.  Because  each  data  point 
represents  a  development  phase,  the  estimate  will  be  of  phase  seven  reliability.  The 
results  of  the  computations  are  shown  in  Table  4. 

Note,  the  sum  of  the  adjusted  trials  is  not  rounded  to  an  integer  value.  Using 
only  the  data  in  the  seventh  phase,  the  MLE  without  discounting  is 
=  7/8  =  .875  . 

The  MLE  using  all  test  data  without  discounting  will  yield 
S-  =  1!.  18  =  .611  . 

For  this  example,  the  MLE  with  discounting  provides  an  improved  estimate  of  actual 
reliability;  i.e.  =  .744  is  closer  to  the  actual  reliability  value,  .78. 

When  testing  is  terminated  at  a  fixed  number  of  failures,  the  MLE  has  an 
additional  shortcoming;  it  is  consen^atively  biased.  Breaking  the  data  into  sets  of  tnals 
up  to  and  including  a  single  failure,  the  number  of  successful  trials  in  a  set  always 


equals  the  number  of  trials  in  a  set  minus  one.  As  such,  the  MLE.  Equation  2.2.  may 
be  stated  as  follows  for  each  set  of  data. 


R  = 


X  -  I 


X 


X  =  N'UMBER  OF  TRIALS  UP  TO  AND  INCLUDING  FAILURE 


The  expected  value  of  5  is  derived  in  Equation  2.9.  Table  5  shows  the  E[H]  compared 
to  the  actual  reliability  R.  Clearly,  the  maximum  likelihood  estimate  will  be  ver>^ 
conservatively  biased  for  low  reliabilities.  [Ref  3:  p.34] 
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E[R}  =  E[(X-1)/X] 

=  1  -  E[  1/X  ] 

E[1,X]  =  (1-R)  -f  (R/2)(I-R)  ^  (R2;3){1-R)  +  ... 

=  1  -  R  -  (R,2)  -  (R-'2)  -f  {R"'3)  -  (R^'3)  ^ 

In(l-R)  =  -R  -  (R-  2)  -(R^;3)  - 
•  { I  Rllni  1  -  R)  =  1  +  R  2  -  R‘ 3  -r-  ... 

.•.E[  1,X]  =  ln[[  -  R)  -  {I;R)ln(l  -  R) 

E[5]  =  1  +  (1  R)ln(l  -  R)  -  ln{l  -  R)  (eqn  2.9) 


TABLE  5 

MLE  EXPECTED  VALUE 

E[S] 

.052 


This  bias  can  be  offset  by  the  proper  choice  of  i  discounting  p.arameter;  but. 
the  development  of  a  less  biased  estimator  would  be  advantageous.  The  E.xponential 
Single  Phase  Estimate,  introduced  in  the  next  section,  demonstrates  a  smaller 
conservative  bias  than  the  MLE  when  applied  to  geometric  data. 
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2.  Exponential  Single  Phase  Reliability  Estimate 

The  Exponential  Single  Phase  Reliability  Estimate  was  developed  for  the 
special  case  of  testing  until  the  fust  failure  is  encountered.  Test  data  of  this  form  has  a 
geometric  distribution  with  the  parameter,  R,  system  reliability.  As  shown  in  the 
previous  section,  the  maximum  likelihood  estimate  is  conservatively  biased  when 
applied  to  geometric  data.  In  an  attempt  to  reduce  this  bias,  an  estimator  of  the 
following  form  was  suggested. 

R  =  1  -  e'^ 

e-A  =  1  -  R 

A  =  -ln(l  -  R) 


I," 


To  complete  the  development  of  the  estimator,  an  unbiased  estimate. 
for  A  was  sought  such  that 
E[^l  =  -ln(l  -  R)  =  A 


> 


The  estimate  is  derived  as  follows.  Let  X.  equal  the  number  of  trials  up  to  and 
including  failure  for  the  i  set  of  geometric  data  and  Rj  equal  the  system  reliability  for 
the  i  set  of  data  where  i  =  1,  2,  3,  ...  ,  number  of  failures.  Assume  that 
Yj  =  f  (  X| )  is  an  unbiased  estimate  of  A.  Then 
x> 

i;f(Xi)  ‘  (l-Rj)  =  -ln(l  -  Rj) 

Xi=  1 

=  R,  +  R ,2  +  R.3  +  ...  . 

Solving  for  f  (  Xj  )  by  equating  coefficients  of  terms  with  like  powers  yields 
fl  X;  )  =  1  +  1/2  -  1,3  +  ...  +  1,(X;  -  1). 

Resulting  in  Equation  2.10. 


k- 


w 


Y;  =  1  +  1/2  +  ...  +l,'(Xi  -  1)  for  Xj  2:  2  (eqn  2.10) 

=  0  for  X;  =  1 

Substituting  Y,  an  unbiased  estimate  of  A,  into  the  reliability  equation  yields  an 
estimate  of  single  phase  reliability,  Equation  2.11.  Equation  2.11  can  be  used  to 
estimate  system  reliability  when  testing  is  performed  until  first  failure. 

As  has  been  shown,  Y  is  an  unbiased  estimate  of  A;  but,  this  does  not  ensure 
Equation  2.11  will  provide  an  unbiased  estimate  of  system  reliability.  To  resolve  this 
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2=1-  e'M 


(eqn  2.1 1) 


question  Monte-Carlo  simulation  was  used  to  evaluate  the  properties  of  the 
Exponential  Single  Phase  Estimate  and  compare  it  with  the  MLE. 

The  algorithm  described  below  was  used  to  evaluate  the  Exponential  Single 
Phase  Estim.ate,  Equation  2.11.  First  a  uniform  (0,1)  random  number  was  generated.  .A 
table  look-up  algorithm  was  used  to  convert  the  uniform  (0,1)  random  variable  to  a 
geometric  random  variable  with  reliability  R.  Estimates  of  system  reliability,  based  on 
the  geometric  random*  variable,  were  computed  using  both  the  Exponential  Single 
Phase  Estimate  and  the  .Vlaximum  Likelihood  Estimate.  The  procedure  was  then 
replicated  a  large  number  of  times.  The  running  average  of  the  estimates  for  each 
estimator  was  updated  after  each  replication.  The  results  of  the  simulation  are  shown 
in  Table  6  for  ten-thousand  replications. 


TABLE  6 

ESTIMATE  EXPECTED  VALUE 
FOR  MLE  &  EXPONENTIAL  ESTI.MATE 


THEORETICAL  RESULTS  SIMULATION  RESULTS 


R 

E[.MLE| 

.MEAN  MLE 

.MEAN  EXP.  EST. 

.1 

.052 

.053 

.066 

•  ^ 

.107 

.107 

.132 

.  j 

.168 

.170 

.207 

.4 

.234 

.234 

.281 

.5 

.307 

.305 

.361 

.6 

.389 

.388 

.452 

1 

.484 

.484 

.552 

.8 

.598 

.598 

.664 

.9 

."44 

.744 

."98 

.99 

.953 

.954 

.968 

.999 

.993 

.994 

.996 

The  simulation  generated  mean  for  the  MLE  is  verv’  close  to  us  theoretically 
derived  value  verifying  the  accuracy  of  the  simulation.  Comparing  the  simulation 
generated  mean  estimate  for  the  Exponential  Single  Phase  Estimator  to  the  actual 
reliability  leads  to  the  conclusion  that  the  estimate  is  ^.onservatively  biased,  but  less 
biased  then  the  MLE.  .As  ;s  the  vtase  .vuh  the  MLE.  tiie  .imcunt  u'  'us  .n  :::c 
Exponential  Single  Phase  Estimate  is  a  function  of  the  actual  reiiabihis  . 

Constant  system  reiiabiiity  is  necessar.'  for  the  Evponentia.  SingiC  Pii-i'e 
Estimate  to  accurately  predict  reliability.  Because  of  this  property,  the  es'imator 
be  modified  to  accurately  predict  the  ..nanging  >V'vtem  reiiaTUit;.  .nnercnt  .r.  'c. 

growth  pattern.  .As  .'.’as  the  .ase  .ntn  tn-  MLE.  'he  mcouwation  r.^u.o 
possible  to  use  ail  available  test  uata  when  computing  a  reiianiif.  -jst.n'.ate.  1  ;.c- 
remainir.g  reliability  grouuh  models  explore  a.tematr.e  mcduiv-a;'.  tro  'o  ■.".c 
Exponential  Single  Phase  Estimate,  file  first  riiodi.'ication  emp.ov^  .u-.e.u-  'ecrev  ..in  ' 
estimate  tiie  parameter  .\.  Tlie  >econd  mcuifiuation  .r.'.':.. es  •.af...''.c  'r.e  .■e.c;'.t--u 
average  of  the  current  •ingie  pnase  estimate  and  th.e  preceding  'itase  'c.i  iPn.f 
to  produce  an  estimate  of  current  pnase  reiiamiuv 

3.  E.xponential  Regression  Discrete  Reliability  Growth  Model 

The  Exponential  Regression  Reliabiluv  Growth  Mode;  was  deseloped  ■'v  W 
.VI.  Woods  .^rom  the  Exponential  Smgle  Phase  Reiiabiiity  Model  [Ref  -H  I  he  mooel 
studied  here  is  identical  to  the  original  model  with  the  exception  cf  applying  fahurc 
discounting.  The  model  can  only  be  applied  :r.  situations  where  testing  ;s  perfcrnieu 
a  fixed  number  of  failures  and  must  have  attribute  data.  .Any  lailure  up..cu:u'.r.g 
method  may  be  used  with  the  model;  but.  the  discounted  failure  data  mu^t  '■'c  re’...r:'.e.: 
to  Its  onginal  form,  an  integer  number  of  failures  and  'UCi.es'es.  K;  •;'.e  oi  ..le.. 

one  must  recoru  "nc  uata  necesvtr.’  to  appr-  tr.e  >noven  .aiiure  u.m..,  urit.n.c  ".c’n.  u.  ’.le 


total  number  cf  failures  and  successes,  and  :r.c  number 
each  failure.  The  equation 
R  =  1  -  e'-'^ 


.  A  —  li  P 

and  k  denotes  the  'esting  pha'^e  is  used  to  compute  s\stetn  'e.;a‘';li". 

R  =  1  -  e-‘«  *  P 
k  =  1.2.  3.  ... 


success;;..;  tria.s  ■'c’-see:. 


Estimating  the  expression  a  Pk,  at  each  phase  allows  the  model  to  track  changing 
reliability.  The  model  is  further  modified  to  include  the  possibility  of  more  than  one 
failure  in  a  given  phase.  Let  Fj^  equal  the  total  number  of  failures  in  the  k  phase  and 
;  equals  the  failure  number  in  phase  k  such  that  j  =  1.2.  ...  F.  equals  '.he 

*  .  p  '  "  U 

numrer  :t  tr.ais  be''.'.  een  the  '■  -  i  ■  and  the  ■  ‘aiiure.  .nciudinc  '.ne  '■  laih.re.  at 


pnuse  ji.  .As  tn  the  hvponentiui  S.nule  Phase  Fi'tiniaie.  Equation  2.!'».  st-.i;  represents 
an  unriaseu  estimate  .u' ■  a  -  p  •.  r  eacit  se'  of  t.'ials  \  To  a  .-tiu  ^...nfm a  t;, 
tne  additicnal  sursennt.  Euuat.on  2  l'<  becomes  f-juation  2.1  2 


tor  .\  >  2 
for  \  =  1 


'  .  -  i  . 


Ls'.nt.ite-  u  mu  P  :.tr  ■'tt.ise  ire  Ue.  el-meu  ;s;nc  'he  ■e.:tn..;t.es  -f  .me. 


e.n  2  1 


'cgress.c: 

.  I 


teldtr.i;  1 


2  lfs  f  a.^at;.  r.  2  I"  requires  tite  ..  m: 


Substituting  the  estimates  tor  and  into  the  reiiabilitv  equation  vieids  the  estimate 
of' reiiabilitv  for  phase  k.  E'quation  2.  IS 


_  -  u  *  p,  k 


-  ei:  r.  2  I 


,\ 

—  *  .1 


A  serar  ite  letmitu  r.  t  r.e  case  .vr.ere  is  =  1  ;s  r.ccessar-  ''e.auNO  reiirevv  •.  e.t.mv.. 


a  lanin.-m  uata  no. ms.  Re:.  pp  i-JJ 

i  oe  otir  m.  q.:Aat:on  masie  m  the  oraiinai  mooe.  ;s  me 


oiscou:v.;.c  Alter  e.;^::  nnu'e.  ’ne  ..itcsen  iaiiure  d;s..oun'me  me'n.) 


o  .'  n.e.. 


.'.e  ! r.is ■ . .i.  .aiiurc'  'i..!:  ■e'-i.-t  mu'-'  "^e  iqm''eo  'o  "s 


"'c  re\.mcO  m  mecer  .  i.'.e  .'.ere:  re.  me  Klji.s’eo  'r:  i.s  .ire  ■  o.rme,;  ■ 

nearest  .ii’eger  f  'tmta'es  ic^r  p.  1. vitiation  2  1'.  and  u.  [  viuatum  2  i',  .ire  ‘i 
^:.mpi.'ev.  A. esc  -an.es  ire  'unsiitmeu  .nt.i  !  qua  turn  2  1s  e'-mi.a’e 

s'  sien.  'e..  mmf.  fir  "'ttvi'C  k. 

Because  ttte  parameters  a  and  P  are  «.omputed  usmt  linear  regressiof;, 
tm  de:  h.is  'ne  .mm''  ■'  ’rack  -tt-ineifii:  reiiabiiit’.  escn  .t  n.t  :am.re  >.  uttt.f.; 


appneq.  !  ten  svi,  ;t  n.at  take  tii.nier.ius  ptiases  and  a  great  den  '!  la’a  h  r  ’l.e 
l.'spctter.'ia.  Regression  \l  .'uci  t..  jv.v.uratei'-  predict  reiubnif.  .\uu:ng  tau  .re 
.i;sv.oi.nm'._'  t  ‘  m.e  m  ale.  'n>  Uid  ie,.rea'c‘  ’ne  imi'  tint  v'l  -la’a  'euvirea  '  '  m  'e 


I  ;e  1  \ ns' tient ..i .  Reg'e- s,  \|.  oe.  ^.in  "'revlie'  t  I'.eoi'.te  -o. -'e::.  te...i''. 
tills  1.^  .rs  :f;.  s:  fen  .n  ttie  ,n:t;  i.  ’est.ng  pnascs  t-.q  at  ha  :e!.a‘'.h'.es  Si.. 
'sciir,  ■  ne  estim.ite  'ti.'u.a  ne  se’  'er.' 


\:.  e's.imn.e  s'.l  hcin 


r.c  mn-.a 't 


_ L  .  .  se  e . . 


A I  te  'ne  't  ^  '..unn  a  is  oe.c'cm  ■'c^.iv.se  ea,.n  nti.ise  >.  i.s.v'v  ;  . 
t  n.ure  ni.i  'i  i  “  ^  tins  ev m.n.e  ,ns.  .lem.  nst:  res  e  rcsa.'s  :  ini'.t 

in.pr  pe'  .to^  '..n’lt.g  pirimeter'  I.t  'ne  s;n..rn  n  I  m  .;e  i.s^  :.c  .se.i  ' 
est:nia'._r  m  :,,r';ier  oterestinute  'i.e  ai.:...:!  re..,f:af.  liao  u;'^  ..m.tm  '"a 
apr.ie.t.  ':.e  'ocressi, n  esmnate  '.s  nnd  h.r.e  '.le.vlen  /?_  =  st( 


The  computations  are  repeated  for  each  phase  of  the  testing  program.  As 
mentioned  previouslv.  when  failure  discounting  alters  the  number  of  successful  trials. 

in  a  preuous  phase,  the  reliability  estimates  for  all  previous  phases  must  be 
recomputed  to  solve  'or  the  current  phase  reliability  estimate.  .As  this  simple  e.xaniplc 
clearl;.  this  reiiubility  grov-.-th  model  is  computationally  intense.  Programmc.ng 

tne  mode,  on  a  computer  may  be  necessary  to  compute  reliability  estimates  for  large 
data  set'. 

Two  I'ailure  aiscounting  methods  and  three  discrete  reliability  growth  m.cdeis 
na'.e  "een  acscrineu  tins  chanter.  Evaluation  of  each  m.odel  is  \'ery  dilficuit  because 
'ne  evp.wte,.  .a.ue  aiiv^  ariance  ri  the  estimates,  when  discounting  is  applied,  become 
mathematmalw  untrai-table.  It  is  a. so  not  possible  to  evaluate  the  models  using  actual 
test  ua’j  necause  tne  true  reliabiiitv  at  each  phase  is  unknown.  The  fcllowing  chapter 
a;'v.U"es  a  M  :nte-<hirm  simit.iat..:n  '.'.r.ttcn  to  c'.aluate  the  pertbrmance  of  each  oi’the 
..i.iarc  ..ntmg  metnous  and  discrete  reaani.itv  gro\sih  models. 


III.  DISCRETE  RELIABILITY  GROWTH  SIMULATION 


Evaluating  reliability  growth  estimators  including  reliabiiity  growth  models,  is 
often  difficult  True  system  reliability  is  unknown,  rendering  actual  test  data  useless  for 
evaluating  an  estimators  performance.  When  the  mathematics  is  tractable,  the 
properties  of  the  estimator  can  be  derived  and  used  to  evaluate  the  estimators 
performance.  However,  many  estimators  are  too  complex  to  allow  mathematical 
solutions  for  the  closed  form  of  their  properties.  For  this  group  of  estimators, 
computer  simulation  is  one  of  the  only  evaluation  techniques  available.  The  addition  of 
failure  discounting  to  an  estimator  makes  mathematical  solutions  very'  difficult  if  not 
impossible.  The  reliability  growth  models  introduced  in  this  study  fall  in  the  categoiy 
requiring  simulation  as  an  evaluation  tool. 

,A  .Monte-Carlo  simulation  was  written  to  evaluate  the  properties  of  the  three 
reliability  growth  models  introduced  in  Chapter  II.  The  simulation  builds  a  reliability 
growih  pattern  and  generates  test  data  from  the  known  system  reliabilies  using  the 
geometric  distribution.  Each  of  the  reliability  growth  models  are  then  applied  to  the 
simulation  generated  test  data.  Repeating  the  process  many  times  for  a  particular  set  of 
known  reliabilities  will  provide  a  good  estimate  of  both  the  mean  and  variance  of  each 
estimator  as  well  as  provide  information  about  the  underlying  distribution  of  the 
estimate.  These  statistics  can  then  be  used  to  evaluate  each  reliability  growth  model's 
performance. 

Several  .medciing  problems  had  to  be  solved  before  the  simulation  could  be  built, 
method  for  generating  a  realistic  reliability  grouih  pattern  had  to  be  developed.  This 
method  must  closely  model  the  process  of  testing  equipment,  implementing  design 
impro’.ements  or  "fi.xes"  and  continued  testing.  It  also  must  providing  the  flexibility  to 
test  numerous  rcliabiiirv  growth  natterns.  Once  the  actual  system  reliability  in  each 
phase  IS  estabiisiied  from  the  growth  pattern,  test  data  of  the  form  the  number  of  trials 
up  to  and  including  ,‘ailure  must  be  generated.  Because  such  data  are  geometrically 
distributed  when  R  is  fixed,  this  process  is  relatively  simple.  .‘\  relistic  method  must  also 
be  developed  for  determuning  the  cause  of  each  failure.  This  step  proved  to  be  the 
most  duTicult. 


As  with  all  simulations,  several  modeling  assumptions  were  necessary.  The 
following  section  discusses  the  assumptions  and  their  impact  on  the  simulation.  The 
remainder  of  the  chapter  describes  the  development  of  the  simulation  algorithm  and  its 
capabilities  and  limitations. 

MODELING  ASSUMPTIONS 

Building  an  algorithm  that  'vill  generate  realistic  test  data  requires  several 
simplifying  assumptions  about  the  actual  test  process.  These  assumptions  can  be 
broken  into  three  categories.  They  are: 

1.  General  assumptions 

2.  .Assumptions  concerning  the  true  reliability  grov.th  pattern 

3.  Assumptions  concerning  failure  cause  determination 

The  last  r.vo  categories  correspond  to  the  two  steps  in  generating  test  data,  generating 
the  actual  reliability  growth  pattern  and  computing  test  data  from  the  growth  pattern. 
Each  assumption  is  listed  and  discussed  by  category. 

1.  General  Assumptions 

The  general  assumptions  narrow  the  scope  of  testing  considered  to  those 
scenarios  which  apply  to  the  reliability  growth  models  evaluated.  The  tw'o  required 
assumptions  are: 

1 .  .All  testing  is  performed  to  a  fixed  number  of  failures. 

2.  All  test  results  are  recorded  as  attribute  data. 

These  assumptions  ensure  the  test  data  produced  by  the  simulation  will  be  compatible 
■with  the  discrete  reliability  growih  models  developed  in  the  study.  The  models  can  only 
be  applied  to  discrete  or  attribute  data  where  testing  is  performed  until  a  fixed  number 
of  failures  are  observed. 

2.  Reliability'  Growth  Pattern  .Assumptions 

Defining  the  mechanisms  which  cause  an  items'  reliability  to  change  from 
phase  to  phase  is  necessary  before  an  algorithm  to  generate  a  realistic  reliability 
growth  pattern  can  be  'vrittcn.  Several  assumptions  were  necessar.'  to  properly  define 
these  mechanisms.  They  are: 

1.  The  reliability  growth  pattern  is  non-decreasing. 

2.  System  reliability  changes  only  at  phase  boundaries. 

3.  Equipment  improvements  are  implemented  immediately  after  a  phase  ends  and 
before  any  further  testing. 
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4.  Only  design  weaknesses  causing  failure  during  a  phase  are  corrected  at  the  end 
of  a  phase. 

5.  Each  design  improvement  removes  a  fixed  fraction  of  the  probability  of 
reoccurrence  for  the  corresponding  failure  cause. 

The  first  assumption  limits  the  shape  of  the  generated  gro\vth  patterns  to  non¬ 
decreasing  curv'es.  Because  one  of  the  objectives  of  development  testing  is  to  improve 
system  reliability,  this  assumption  is  logical.  However,  actual  reliability  growth  may 
have  periods  of  decline  when  "fixes"  to  design  problems  actually  decrease  reliability. 
Over  the  entire  testing  period,  even  these  curves  are  generally  Increasing.  The  worst 
case  this  assumption  allows  is  no  growth  over  the  testing  phases.  This  should  be 
adequate  for  determining  adverse  growth  characteristics  of  the  reliability  growth 
models. 

.Assuming  system  reliability  remains  constant  throughout  a  testing  phase 
makes  the  model  consistent  with  the  previous  definition  of  a  testing  phase.  This 
assum.prion  means  no  design  changes  or  any  other  improvements  are  made  to  an  item 
on  test  within  a  phase.  Because  a  phase  can  be  defined  as  any  number  of  failures 
greater  than  or  equal  to  one,  this  assumption  should  have  little  impact  upon  the 
accuracy  of  the  simulation. 

Assuming  all  fixes  are  implemented  after  a  test  phase  ends  and  before  the  next 
phase  begins  is  closely  related  to  the  previous  assumption.  This  assumption  does 
preclude  accurate  modeling  of  long  term  fixes  whose  implementation  are  postponed 
while  testing  continues.  The  impact  of  this  limitation  should  be  minimal. 

.Assumption  four  asserts  that  the  only  known  weaknesses  in'  system  design  are 
these  which  have  caused  system  failure.  This  is  not  always  the  case.  Suspected 
weaknesses  may  be  corrected  along  with  those  weaknesses  confirmed  by  system  failure. 
The  simulation  is  unable  to  model  this  situation.  Even  though  this  deviates  somewhat 
from  reality,  most  major  improvements  to  system  reliability  should  come  from  failure 
identified  weaknesses.  The  intent  of  the  simulation  is  only  to  generate  a  realistic 
reiiability  growth  pattern  not  to  duplicate  an  actual  growth  pattern.  Therefore  the 
simulation  results  should  be  adequate  to  accomplish  the  task  for  which  the  model  was 
designed. 

The  last  assumption  in  this  category  deals  with  the  mechanism  used  by  the 
simulation  to  model  improvements  in  system  design.  It  asserts  that  all  fixes  remove  a 
constant  fraction  of  the  probability  of  occurrence  for  the  failure  cause  of  interest.  In 
reality  many  factors  determine  how  effective  a  particular  fix  will  be  and  all  fixes  are  not 


equally  effective.  This  method  also  decreases  the  probability  of  occurrence  more  for  a 
failure  cause  with  a  high  probability  of  occurrence  than  for  a  failure  cause  with  a  low 
probability  of  occurrence.  This  also  is  unrealistic.  Despite  the  shortcomings  of  this 
simplification,  it  does  provide  a  simple  way  to  model  reliability  improvement  due  to 
design  change  and  will  produce  a  growth  pattern  that  is  adequate  for  evaluating  the 
reliability  growth  models  studied. 

The  final  category  of  assumptions  deal  with  generating  test  data  from  a  known 
system  reliability  and  determining  the  cause  of  each  failure.  They  are  discussed  in  the 
next  section. 

3.  Failure  Cause  Determination  Assumptions 

Realistically  modeling  failure  causes  is  critical  to  the  evaluation  of  the  failure 
discounting  methods.  The  technique  used  to  model  failure  cause  determination  must 
also  be  incorporated  into  the  technique  used  to  model  reliability  growth  because  the 
two  processes  are  closely  related.  The  assumptions  made  in  modeling  .fiiilure  cause 
determination  are: 

1.  There  exists  a  finite  number  of  possible  failure  causes. 

2.  Each  failure  cause  has  a  fixed  probability  of  occurrence  in  each  phase. 

3.  System  reliability  can  be  modeled  as  a  series  system  of  the  failure  causes. 

4.  Each  failure  cause  is  stochastically  independent  of  the  other  failure  causes. 

Assuming  a  fixed  number  of  failure  causes  may  be  an  oversimplification  of 
reality  but  should  not  invalidate  the  model.  Even  though  the  possible  number  of  failure 
causes  is  most  likely  infinite,  only  a  finite  number  of  these  causes  with  relatively  high 
probabilities  of  occurrence  are  of  interest.  The  number  of  possible  failure  causes 
considered  by  the  simulation  must  be  specified  as  an  input  to  the  model;  but,  the 
number  of  causes  input  is  only  limited  by  the  capacity  of  the  computer  on  which  the 
model  is  run.  Therefore,  the  simulation  is  fle.xible  enough  to  model  nearly  any  scenario 
of  interest. 

Fixing  the  probability  of  occurrence  for  all  failures  causes  within  a  single 
phase  is  a  direct  result  of  fixing  the  system  reliability  for  all  trials  in  a  single  phase. 
This  assumption  will  not  further  restnct  the  ability  of  the  algonthm  to  model  reality. 

An  assumption  is  needed  to  interrelate  failure  cause  determination  and  system 
reliability  growth  through  design  improvements.  .Assuming  the  component  being  tested 
may  be  modeled  as  a  series  system  of  failure  causes  meets  this  requirement.  System 
reliability  at  any  phase  is  a  function  of  the  probability  of  occurrence  for  the  specified 


failure  causes.  As  such,  implementing  a  design  improvement  reduces  the  probability  of 
occurrence  for  a  failure  cause  which  in  turn  increases  the  systems  reliability.  This 
assumption  also  makes  it  possible  to  stochastically  determine  the  cause  of  failure.  The 
mathematical  derivation  of  this  technique  will  be  presented  in  a  later  section. 
Modeling  the  failure  cause  as  a  series  system  does  not  depart  far  from  reality  because 
the  occurrence  of  a  failure  cause  implies  system  failure.  Once  the  assumption  of  a 
finite  set  of  failure  causes  is  accepted,  modeling  them  as  a  series  system  does  not 
further  restrict  the  models  representation  of  reality. 

The  final  assumption  requires  that  each  failure  cause  be  stochastically 
independent.  This  means  that  the  causes  of  failure  are  not  related.  The  occurrence  of 
one  cause  does  does  not  influence  the  probability  of  occurrence  for  any  other  cause.  If 
the  failure  causes  are  properly  defined,  this  assumption  should  not  limit  the  accuracy  of 
the  model. 

The  following  section  discusses  the  simulation  algorithm  built  from  these 
assumptions. 

B.  SIMULATION  ALGORITHM  DEVELOPMENT 

The  algorithm  developed  for  this  simulation  can  be  broken  into  the  four  sections. 
They  are: 

1.  Generate  a  reliability  growth  pattern. 

2.  Generate  test  data,  including  the  cause  of  failure,  from  the  reliability  growth 
pattern. 

3.  Apply  the  failure  discounting  methods  and  reliability  growth  models  to  the 
simulation  generated  data. 

4.  Compute  statistics  used  to  evaluate  the  performance  of  each  model. 

The  lirst  two  steps  of  the  algorithm  along  with  the  output  statistics  will  be 
discussed  in  detail  in  this  section.  The  methodology  used  to  apply  each  of  the 
discounting  methods  and  reliability  growth  models  was  discussed  in  Chapter  II  and  will 
receive  little  attention  here.  With  few  exceptions,  the  actual  content  of  the  Fortran 
code  used  in  the  simulation  will  not  be  discussed.  .A  documented  copy  of  the  code  'is 
included  at  .Appendix  .A  for  the  interested  reader. 

1.  Simulation  of  a  Reliability'  Growth  Pattern 

The  reliability  growth  pattern  built  by  the  simulation  models  the  actual 
process  of  testing  and  improving  an  item  in  a  development  program.  Several  user 
inputs  are  required  at  this  stage  of  the  simulation.  They  are: 

I.  number  of  testing  phases 


2.  number  of  failures  in  each  testing  phase 

3.  number  of  possible  failure  causes 

4.  the  first  phase  probability  of  occurrence  for  each  possible  failure  cause 

5.  growth  fraction  (F^) 

Once  these  parameters  are  in  place,  the  simulation  computes  the  first  phase 
system  reliability  from  the  probability  of  occurrence  for  each  possible  failure  cause.  The 
number  of  trials  up  to  and  including  failure  and  the  failure  cause  are  then  computed  for 
all  failures  in  the  first  phase.  The  algorithm  used  to  generate  system  reliability  and  the 
failure  data  will  be  covered  in  the  next  section. 

.After  the  failure  data  for  the  first  phase  has  been  generated,  the  probability  of 
occurrence  for  each  failure  cause  which  occurred  in  the  first  phase  is  recomputed  using 
the  growth  fraction.  .Assume  the  first  failure  occurred  due  to  cause  .A,  the  new 
probability  of  occurrence.  Pr  (occurrence)^,  will  be  computed  using  Equation  3.1. 

Pr  (occurrence)^  =  Pr( occurrence)  x  (1  —  F^)  (eqn  3.1) 

This  equation  is  applied  at  the  end  of  each  phase  to  those  failure  causes  that  occurred 
in  the  current  phase.  If  a  failure  cause  is  responsible  for  multiple  failures  in  the  phase. 
Equation  3.1  is  applied  only  one  time.  This  procedure  simulates  identifying  a  system 
weakness  through  testing  and  implementing  a  design  improvement.  The  procedure  is 
repeated  at  the  end  of  each  testing  phase. 

The  result  of  the  algorithm  is  a  stochastically  determined  reliability  growth 
pattern  that  simulates  actual  developmental  testing.  .A  test  is  performed,  weaknesses  in 
the  system  are  identified,  corrective  design  changes  arc  implemented  and  testing  is 
continued.  This  algorithm  also  has  the  advantage  of  interrelating  the  probabilities  of 
occurrence  for  each  failure  cause  and  the  system  reliability.  This  will  be  useful  in 
determining  the  failure  cause.  By  changing  the  growth  fraction,  the  rate  of  reliability 
grow'th  can  be  varied  from  no  growth  to  rapid  growth. 

2.  Generating  Te.st  Data  From  Known  Reliability 

Two  problems  must  be  solved  before  an  algorithm  that  generates  test  data  can 
be  developed.  The  system  reliability  must  be  determined  from  the  probability  of 
occurrence  for  each  failure  cause.  .A  technique  for  generating  the  number  of  trials  up  to 
and  including  failure  from  known  reliability  that  identifies  the  failure  cause  also  must 
be  developed.  The  solution  to  both  problems  lies  wath  the  assumption  that  system 
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reliability  can  be  modeled  as  a  series  system  of  the  failure  causes.  In  a  discrete  series 
system  the  number  of  trials  up  to  and  including  system  failure  will  be  the  minimum  of 
the  number  of  trials  up  to  and  including  failure  for  each  sub-component.  Let  M  equals 
the  number  of  trials  up  to  and  including  failure  for  the  system  and  Xj  where 
i  =  1  ,2  ,  ...  ,  i  denotes  the  number  of  trials  up  to  and  including  failure  for  the  i 
subcomponent,  here  a  failure  cause.  If  the  j  components  form  a  series  system,  M  may¬ 
be  expressed  as 

M  =  MIN  (XpX.,...,X.} 

Because  the  probability  of  occurrence  is  fixed  throughout  a  phase  for  each  failure 
cause.  Xj  ;s  distributed  as  a  geometnc  random  vanable  with  parameter  R..  From  an 
assumption,  the  X/s  are  also  mutually  independent.  It  can  be  shown  for  a  geometnc 
random  variable,  Xj,  that 
PriX;  >  ri)  =  R;''. 

The  distribution  for  M  is  derived  as  follows. 

M  =  MIN  [  Xi,X,....,Xj  1 
Pr  [M  >  n]  =  Pr  [  Xj  >  n,  X2  >  n, ...  .Xj  >  n  ] 

=  Pr  [Xj  >  n]  X  Pr  [X2  >  nj  x  ...  x  Pr  [Xj  >  n] 

=  R X  R.,"  X  ...  X  R.n 

1  2  J 

=  [  X  R,  X  ...  X  R.  jn 

Therefore  M  is  also  distributed  as  a  geometric  random  variable  with  the  parameter 

R  =  (  Rj  X  R^  X  ...  X  Rp  . 

The  system  reliability  at  each  phase  is  computed  as  the  product  of  the 
compliments  of  the  probabilities  of  occurrence  for  each  possible  failure  cause.  Because 
of  this  computation  the  actual  simulation  input  is  the  compliment  of  the  first  phase 
probability  of  occurrence  for  each  failure  cause.  .As  the  probability  of  occurrence  for 
each  failure  cause  is  recomputed  from  phase  to  phase,  the  system  reliability  is  also 
adjusted. 

The  number  of  trials  up  to  and  including  failure  for  a  geometrically  distributed 
random  variable  with  reliability  R  can  be  generated  from  a  uniform  (0.1)  random 
vanable.  U,  using  Equation  3.2.  [Ref.  5]. 


X  =  INT  [  1  +  (  ln(U)/In(R) ) 


(eqn  3.2) 


Equation  3.2  combined  with  the  definition  of  a  series  system  provide  the  solution  to 
generating  the  number  of  tnals  up  to  and  including  failure  w'hile  accounting  for  the 
cause  of  failure. 

For  each  failure  generated  by  the  simulation.  Equation  3.2  is  used  to  generate 
a  geometric  random  variable  for  each  failure  cause  using  the  compliment  of  me 
probability  of  occurrence  as  the  parameter.  The  minimum  geometric  random,  vananie 
over  the  failure  causes  becomes  the  number  of  tnals  up  to  and  including  lailure  for  the 
system.  The  cause  of  system  failure  is  recorded  as  the  failure  cause  producing  tlie 
minimum  geometric  ranucm  variable.  Generating  the  test  data  in  this  lashion  allows 
the  reliability  grownn  pattern  and  test  data  to  be  stocnasticaily  generateu  yet 
interrelated  much  as  they  are  in  nature.  A  summary  of  the  steps  required  to  compute 
the  reliability  growth  pattern  and  the  test  data  follows. 

1.  Input  the  required  data 

2.  Generate  phase  failure  data  including  the  number  of  tnais  and  the  failure  cause. 

3.  Recompute  the  probability  of  occurrence  for  the  failure  causes  that  occur  :n  the 
phase. 

4.  Return  to  step  2  until  ail  phases  are  complete. 

Once  the  test  data  has  been  computed  from  the  known  reliabilit;.'  in  each 
phase,  the  failure  discounting  methods  and  reliability  growth  models  can  be  applied  to 
the  test  data.  TTie  resulting  reliability  estimates  can  then  be  evaluated  against  the 
known  system  reliability. 

3.  Application  of  the  Discounting  Methods  and  Reliability  Growth  Models 

The  discounting  methods  and  reliability  growth  models  are  applied  to  the 
generated  test  data  in  much  the  same  way  as  they  were  described  in  Chapter  II.  Each 
of  the  three  reliability  growth  models  are  applied  to  each  set  of  test  results.  However, 
the  simulation  user  must  specify  one  of  the  two  failure  discounting  methods  because 
the  simulation  can  not  apply  both  simultaneously.  The  ma.ximum  likelihood  estimate 
without  discounting  is  also  computed  for  each  phase,  independent  of  all  other  test  data, 
This  allows  a  comparison  between  the  standard  reiiabiluy  estimator  and  the  .'cliapiiitv 
growth  models.  The  simulation  produces  an  estimate  of  system  reiiabiiity  iVom  eacit 
reliability  growih  model  for  each  testing  phase.  The  single  phase  .\1LE  and  actual 
system  reliability  are  provided  as  a  basis  for  comparison. 

One  minor  departure  from  the  application  of  the  reliability  growih  models  as 
described  in  Chapter  II  was  necessary'.  For  large  numbers  of  trials  up  to  and  including 
failure,  \,  computations  of  the  form 


Y  -  I  -  (1  2)  ^  ...  +  [1  (N-l!) 

required  extensive  computer  processing  time  to  solve  recursively.  To  improve  program 
efTiciency.  the  values  for  Y  up  to  and  including  one-thousand  were  computed  exactly 
and  stored  in  an  array.  For  values  of  \  greater  than  one-t.,ousanJ  non-linear 
regression  ivas  usee  to  estimate  the  value  .of  'i'.  huuation  prov;ded  an  adequate 
estimate  wuh  an  R*  value  cf 


X  N^  Ul  100 1 


lean  3.3; 


CCMFARISJN  C~  AC'^A_  VS  ES'MA^uS  VAi_jES  Y 


Figure  3.1  Regression  Estimate  ofY. 

Figure  3.1  compares  the  actual  Y  value  with  the  regression  predicted  value  for  large  N. 
Both  the  Weighted  Average  .Model  and  the  E.xponential  Regression  Model  are  affected 
by  the  approximation.  The  approximation  has  minimal  impact  upon  the  computation 
of  model  e^r;mates.  Values  for  N  above  one-thousand  are  rare  even  at  high  reliability 
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and  values  above  15. 'XH),  the  accurate  extension  of  the  approximation,  are  verx  rare 
Should  a  value  of  \  larger  than  1‘'.iXK)  occur,  some  error  ’.vil!  he  introduced  into  the 
model  estimate,  and  this  error  will  increase  vvith  N  Because  these  values  are  rare,  this 
error  is  not  considered  signit'icar.t 
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Statistical  Summars  oi  \lodei  Results 


Several  descriptive  stat 
the  perl'ormance  of  each  reiiabf 


istics  are  generated  by  the  model  that  aid  in  evaluating 
litv  growth  medei.  They  include  the  mean  and  standard 


deviation  of  the  reliabiiity  estimates  at  each  phase  for  each  of  the  growth  models.  .A 
confidence  interval  is  also  computed  tor  the  mean  estimate  at  each  phase. 

Replications  of  the  smtuiatior.  are  required  to  compute  the  descriptive 
statistics.  .After  the  first  replication,  the  actual  system  reliability  m  each  phase  is  fixed. 
•At  each  'epucntion,  new  test  data  .s  generated  from  the  fixed  reliability  grouth  pattern 


and  oa^n 
iCr.cr.d ; 


the  renabiiitv  growrn  mcucis  is  applied  to  the  uata.  The  estimates 
me  gr.tw'n  rrismcxv  .ire  rcainec  at  each  replication.  Wiien  the  ocsired 


number  ol  replications  have  been  completed,  the  sampie  mean  and  sample  standard 
deviation  for  eaeh  estimator  are  computed  using  standard  formulas.  .A  *^5'’  >  confidence 


interval  .or  'he  mean  estimate  .s 


computed  lor  each  phase  reliabilitv  estimate  .vith.  tne 


assun.nti.m  mat  e.wn  .■■.timate  .s  normally  uistnoutcu.  T'n.c  assumption  of  n.crmaatv  .s 
not  va..u  in  every  case  nut  the  confluence  interval  is  inc.uced  as  a  tool  for  comparison. 


Further  analysis  of  the  reliability  growth  models  can  be  accomplished  by  saving  eaLh 
estimate  in  an  output  file.  The  models  user  can  then  anah^c  the  data  as  desired. 

Changing  the  simulations'  input  parameters  makes  it  possible  to  analyse  a 
wide  variety  of  testing  situations.  These  capabilities  are  discussed  in  the  following 


section. 


c.  capabilities  of  the  model 


Within  the  limits  of  the  assumptions  stated  earlier,  the  simulation  is  ver>'  flexible. 
Any  number  of  failure  causes  may  be  input  into  the  model.  By  varying  the  phase  one 
probability  of  success  for  each  failure  cause,  the  grouih  fraction  and  the  number  of 
phases  a  '.v-.de  range  of  rehability  growth  curves  .may  be  explored.  The  .'anaom  number 
generator  seed  is  aiso  user  specified  uilowmg  the  exact  string  of  ranuent  numbers  anu 
test  data  to  be  duplicated.  In  this  way.  va.nous  discounting  .methods  and  parameters 
may  be  evaluated  using  tire  exact  -.ame  data. 

The  test-fiX-W'st  ^cenano  may  be  modeled  .sy  specil'ying  a  single  failure  per  pnase. 
n.e  te:  i-f.nd'tes:  .metnouoiogy  may  ".e  Mmuiaied  i:y  spccilying  muitipie  failures  ,n  eacii 
phase.  There  is  no  requirement  mat  each  phase  have  an  equal  number  of  laiiures. 


Therelcre  a  maxed  scenario,  test-iix-test  and  test-nnd-test.  may  be  simulated. 

Ic.wn  of  the  discounting  methods  aesenbeu  in  Chapter  II  may  be  applied  to  the 
uata  '.'^:iig  me  .'u.I  mnge  of  .nput  parameters.  Proper  parameter  -election  for  me 
discounting  metr.cu  wdi  cause  no  uiscounting  to  occur.  In  this  manner  each  of  tne 


reiiabiiity  growth  m.oceis  may  be  evaluated  without  failure  discounting  applied. 

The  s.mulation  proMdes  a  tool  for  evaluating  the  reliability  growxh  models 
against  a  known  system  reliability  growth  pattern.  When  large  numbers  of  replications 
are  run,  information  concerning  the  expected  value  and  vanance  of  each  estimator  may 
be  obtained.  The  combinations  of  failure  discounting  parameters,  reliability  grow-th 
models  and  anticipated  growth  patterns  may  also  be  evaluated.  The  following  chapter 
discusses  the  results  of  numerous  runs  of  the  simulation  with  varying  growth  patterns 
and  failure  discounting  parameters. 


IV.  EV ALL ATION  OF  RELIABILITY  GROWTH  MODELS 


The  Jiscou.nting  methods  and  reliability  growth  models  were  evaluated  using  hie 
Monte-Carlo  simulation  described  m  Chapter  Ill.  The  results  from  numerous  runs  n' 
the  simulation  with  '.aremg  reliability  growih  patterns,  failure  discounting  methods  and 
failure  discounting  param.eters  were  reduced  to  graphical  output  for  evaluation. 

The  purpose  of  'he  evaluation  was  to  determine  general  characterntijs  o!  e.i..:t 
reliability  growth  .model.  The  number  of  simulation  runs  was  not  adequate  to  prove 
that  conclusions  reached  here  will  held  for  all  cases.  Also,  no  fast  rules  for  model 
selection  or  failure  discounting  parameter  selection  were  developed.  Still,  mans 
characteristics  cl'  the  diirereut  .'eiiabiiity  growth  models  were  apparent  ‘fem  'f.e 
si.miuiation  runs  comoieted.  In  particular  the  answers  to  tlie  fcilcwma  cuesticns  were 


sought. 

1.  How  well  does  the  reliability  growth  model  track  the  actual  reliability  growth 
pattern? 

2.  How  did  the  choice  of  failure  discounting  parameters  alTect  the  estimates  of 
reliability? 

3.  How  variable  were  the  estimates  provided  by  each  of  the  models'* 

It  was  impossible  to  evaluate  all  possible  reliability  grovsth  patterns  that  can  be 
produced  by  the  simulation.  Five  widely  different  reliability  growth  patterns  were 
chosen  to  be  representative  of  the  possible  range.  Constant  reiiabiiity  cases  were 
included  to  identify  m.odels  that  predict  reliability  growth,  were  m  fact,  none  c.\.ists.  l  i'.c 


five  reliability  growth  patterns  chosen  are  described  below. 

1.  low  reliability  with  no  growth 

2.  high  reliability  with  no  growth 

3.  low  initial  reliability  wth  moderate  growth 

4.  moderate  mitia.  reiia''f..tv  .vitn  moderate  growth 

5.  low  initial  rciiabilr.s  witii  rapid  growth 

.All  simulation  input  parameters  were  kept  fixed  to  insure  the  dilTerent  simulation 
runs  could  be  compared.  The  number  of  phases  was  held  constant  at  ten  with  one 
failure  per  phase.  Five  failure  causes  were  used  and  each  simulation  was  replicated  one- 
hundred  times.  The  seed  lor  the  random  number  generator  was  kept  constant  for  all 
simulation  runs  insuring  the  same  string  of  random  numbers  and,  therefore,  identical 
test  data  was  generated  for  each  growih  pattern. 
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fiach  rehabiluy  grouth  mode!  is  evaluated  separate!’,  in  'he  lcl!ovvir.g  section.'' 
The  graphical  results  ol'thc  remaining  simulation  runs  are  irmiuded  in  Appendix  B 
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Figure  4.1  Reliability  Growth  Models 
Low  Constant  Reliability 
No  Discounting  .Applied. 


The  choice  of  failure  discounting  method  and  parameter  is  critical  to  the  .MLE 
W;:h  Di'^counting  Model.  Because  the  Lloyd  Discounting  Method  does  not  have  a 
disccunting  .nter'.ai.  it  generally  reduces  the  discounted  failure  value  more  rapidly  than 
the  Straignt  Percent  Discounting  Method.  This  results  in  rapid  growth  of  the  reliability 
estimates.  The  Standard  Discounting  Metnod  wath  the  discount  interval  set  at  one  and 
the  proper  choice  of  the  discount  fraction  wall  provide  nearly  identical  results  to  the 
Llcvd  m.ethod.  For  all  ’he  reliahilitv  anwtn  nattcrns  tested,  exceot  low  initial  reliabilitv 


with  rand  crowrn.  'he  L.ovd  discounting  ^le:hod  caused  "he  model  to  overestimate  the 


actual  reiiahiiity.  The  overestimation  was  particularly  pronounced  for  low  constant 
rehabiiif.  where  the  Lloyd  Discounting  .Method  forced  the  model  to  overestimate 
reliabilit'.'  even  with  y  as  large  as  .99999.  Figure  4.".  Because  of  this  characteristic,  the 
Lloyd  discounting  Method  is  not  recommended  for  use  with,  the  Ma.ximum.  Likelihood 
Est. mate. 
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Figure  J.2  Reliability  Growth  Models 
Low  Constant  Reliability 
Std.  Discounting  .Applied. 

With  the  addition  of  the  second  parameter,  the  discount  interv'al,  the  Straight 
Percent  Discount  Method  is  more  flexible  than  the  Lloyd  Method.  Even  so.  it  has  the 
potential  to  overestimate  the  system  reiiabiiity.  The  choice  of  model  parameters  is 
cr.ticai  to  the  model’s  pericrmance.  Figure  U.i  demonstrates  the  change  in  m.odei 
estimiates  with  vaiying  discounting  parameters.  Several  factors  inlluence  the  proper 
choice  of  parameters.  Three  of  the  most  important  are  the  am.ount  of  testing  to  be 
performed,  estimates  of  system  reiiabiiity  bc.^ore  testing  begins  and  how  rapidly 
reliabiiitv  is  expecteu  to  grow  throughout  the  development  and  testing  program.  No 
ruies  have  been  deveiopeu  to  .heip  seiect  the  proper  discounting  parameters.  E.xper.ence 
with  the  model,  good  engineering  judgement  and  possibly  simulation  are  necessary  to 
select  a  combination  of  discounting  parameters  that  will  cause  the  modei  to  accurately 
track,  the  actual  reliability  growth  pattern.  Even  an  apparently  good  choice  ot  failure 
discounting  parameters  may  resuit  in  erroneous  reliability  estimates  if  the  true 
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Figure  4.3  Reliability  Growth  Models 
High  Constant  Reliability 
Std.  Discounting  .Applied. 

reliability  does  not  grow  as  expected.  Therefore,  when  using  this  model,  frequent 
engineering  evaluations  may  be  necessary’  to  insure  design  evolution  is  progressing  as 
expected. 

Reliability  estimates  from  the  .Maximum  Likelihood  Estimate  With  Discounting 
Model  exhibited  the  smallest  variance  of  the  models  evaluated.  Figures  4.9  and  4.10 
are  box  plots  of  the  one  hundred  reliability  estimates  at  each  testing  phase  for  two  of 
the  reliability  growth  patterns.  Comparison  '.\ith  other  models  may  be  made  from  the 
additional  box  plots  included  at  .Appendix  B.  The  reliability  estimates  from  all  three 
models  exhibited  large  variance  in  the  first  few  phases  of  testing.  The  variance  of  the 
estimates  gradually  decreased  as  more  test  data  became  available.  With  few  exceptions, 
the  MLE  With  Discounting  exhibited  the  smallest  variability  over  the  entire  range  of 
phases  for  each  reliability  growih  pattern  tested.  This  reduced  variance  was 
panicularly  pronounced  in  the  early  phases  and  at  low  reliability. 


51 


STD  DISCOUNTING.  N  =  10  ;  F  =  .75 


o 


Figure  4.4  Reliability  Growth  Models 
Low  Reliability  Moderate  Growih 
Std.  Discounting  .Applied. 

The  MLE  With  Discounting  Model  can  accurately  track,  most  reliability  growth 
patterns  when  the  Straight  Percent  Discounting  Method  is  used.  It  does  so  with  equal 
or  less  variance  than  the  other  models  evaluated.  However,  the  model  is  very  sensitive 
to  the  choice  of  of  failure  discounting  parameters  and  may  either  over  or  under 
estimate  the  actual  reliability  if  the  parameters  are  not  chosen  correctly. 

D.  EXPONENTIAL  REGRESSION  MODEL 

The  E.xponentiai  Regression  .Vlodel  has  the  ability  to  accurately  track  most 
reliability  growth  patterns.  Because  it  uses  a  combination  of  linear  regression 
techniques  and  failure  discounting  to  track  changing  reliability,  it  reacts  quickly  to 
changes  in  actual  reliability.  Figures  4.2,  4.3,  4.4,  4.3,  and  4.6  demonstrate  the  model  s 
performance  with  each  grovnh  pattern.  It  also  is  much  less  sensitive  to  the  selection  of 
failure  discounting  parameters  than  is  the  MLE  With  Discounting  Model.  However, 


Figure  4.:>  Reliability  Grovuh  Models 
High  Reliability  Moderate  Grcuth 
Std.  Discounting  Applied. 

the  variance  of  the  early  phase  estimates  is  much  greater  than  the  estimates  from  the 
MLE  'With  Discounting  Model, 

This  model  requires  several  phases,  usually  at  least  four,  to  accurately  estim.ate 
system  reliability.  The  number  of  phases  required  is  a  function  of  the  shape  of  the 
reliability  groivth  pattern.  More  phases  are  required  for  an  accurate  estimate  at  low 
system  reliability  than  at  high  reliability.  The  model  also  appears  to  track  changing 
reliability  more  accurately  than  constant  reliability. 

The  Exponential  Regression  Mode;  is  relatively  robust  with  respect  to  the 
selection  of  a  failure  discounting  method  and  parameters.  Still,  as  was  the  case  with  the 
MLE  With  Discounting  Model,  the  Lloyd  Discounting  Method  tends  to  cause  the 
model  to  overestimate  actual  reliability  as  is  demonstrated  by  Figure  4.11.  Figure  4.12 
demonstrates  how  robust  the  model  can  be  to  changing  parameters  when  the  Straight 
Percent  Discount  Method  is  applied.  Because  the  Straight  Percent  Discounting 
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Figure  4.6  Reliability  Grow'th  Models 
Low  Reliability  Rapid  Growth 
Std.  Discounting  Applied. 

Method  can  produce  nearly  equivalent  results  and  is  much  more  flexibie,  use  of  the 
Lloyd  Discounting  Method  is  not  recommended  with  this  model. 

Again,  no  fixed  rules  exist  for  chosing  the  failure  discounting  parameters  of  the 
Straight  Percent  Discounting  Method.  Expenence  with  the  mode:,  engmeenn 
judgement  and  possibly  computer  simulation  will  be  necessary'  to  select  the  proper 
combination  of  the  discount  interval  and  discount  fraction.  Even  though  the  model  is 
fairly  robust  concerning  this  selection,  it  is  possibie  to  choose  parameters  that  will 
force  the  model  to  overestimate  reliability.  The  danger  of  this  occurring  is  greatest 
when  expected  growth  in  the  system  reliability  does  not  occur. 

The  reliability  estimates  produced  by  the  exponential  regression  model  exhibit  a 
higher  variance  than  those  from  the  MLE  With  Discounting  Model.  This  increase  in 
variance  is  most  pronounced  in  the  early  testing  phases.  Figures  4.13  and  4.14  are  box 
plots  of  the  one-hundred  reliability  estimates  at  each  phase.  They  demonstrate  the 
variability  of  the  model. 


54 


<rQ 


o 

o 


MLE  WITH  DISCOUNTING 


—  ACTUAL  RELIA8IUTY 
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Figure  4.7  MLE  With  Discounting 
Lloyd  Discounting  Method. 

As  vvith  any  regression,  several  points  are  required  to  stabilize  the  regression 
parameter  estimates.  This  causes  highly  variable  responses  in  the  first  two  or  three 
phases  of  testing.  This  variance  is  largest  when  reliability  in  the  early  phases  is  low.  In 
later  phases  and  at  higher  reliability,  the  variance  of  estimates  from  the  Exponential 
Regression  Model  compares  favorably  to  those  of  the  .MLE  With  Discounting  Model. 

The  E.xponential  Regression  Model  closely  tracks  most  reliability  growth 
patterns.  It  is  best  when  more  than  three  test  phases  are  planned.  The  most  accurate 
results  were  obtained  when  the  model  was  used  with  the  Straight  Percent  Discounting 
Method.  The  model  is  relatively  robust  with  respect  to  the  selection  of  discounting 
parameters;  but,  may  still  over  or  under  estimate  actual  reliability  if  a  bad  combination 
of  parameters  is  selected.  The  variance  of  reliability  estimates  in  the  early  testing 
phases  is  large  and  makes  the  model  undesirable  for  use  with  less  than  four  testing 
phases. 
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Figure  4.8  MLE  With  Discounting 
Straight  Percent  Discounting  Method. 
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4.11  E.xponential  Regression  Model 
Lloyd  Discounting  Method. 


V.  SUMMARY  AxND  CONCLUSIONS 


A.  SLMMARY 

The  purpose  of  this  study  is  to  develop  and  evaluate  discrete  reliability  growth 
models  that  emplo>  iractionai  failure  reduction,  referred  to  as  failure  discounting. 
These  models  use  ail  available  data  to  estimate  current  reliability  even  if  the  actual 
reliabilitv  underlving  ail  test  data  is  not  constant.  This  reduces  the  resources  required 
to  veriiy  system  reliability.  This  study  focuses  on  developmental  testing  scenarios  where 
testing  IS  terrrunated  after  a  fi.xed  number  of  failures  have  been  observed  and  the  results 
of  each  trial  are  recorded  as  a  success  or  failure,  i.e.  attribute  data. 

Failure  discounting  is  performed  on  past  data  to  make  it  compatible  with  test 
data  taken  from  'he  rurrent  system  configuration.  Most  reliability  estimators  require  a 
constant  actual  reiiaciiity  for  eacn  trial  to  render  an  accurate  estimate,  i^ractionaily 
discounting  past  failures  adjusts  test  data  from  previous  testing  on  less  reliable  system 
configurations  making  the  data  useful  in  the  computation  of  the  current  reliability 
estimate. 


Two  different  failure  discounting  methods  were  used  in  the  study.  The  first, 
referred  to  as  the  Straight  Percent  Discounting  Method  allows  the  user  to  predetermine 
the  fraction  by  which  each  failure  will  be  reduced,  the  discount  fraction,  and  an  interval 
between  applications  of  the  discount  fraction,  the  discount  interval.  The  discount 
interval  is  the  number  of  trials  without  a  reoccurrence  of  the  particular  failure  cause 
between  application  of  the  discount  fraction.  The  second  failure  discounting  method, 
referred  to  us  the  L!o;.d  Discounting  .Method,  sets  all  past  failures  equal  to  the  upper 
confidence  bound  for  the  probability  of  reoccurrence  for  the  failure  cause.  Both  failure 
discounting  methods  were  used  with  each  of  the  three  discrete  reliability  growth  models 


!  liree  di'.crete  reiiabiiitv  growth  modeis  using  failure  irwounting  were  developed. 
The  lirst  model  adds  laiiurc  discounting  to  the  maximum  likelihood  estimate  for  a 
proportion,  file  model  is  referred  to  as  the  Maximum  Likelihood  Lstimate  \V'!th 
Discounting  Model,  To  use  tlie  model,  the  failure  discounting  method  of  choice  is 
applied  0  the  test  data.  The  resulting  discounted  failures  are  returned  to  unity  hy 
ad,usf.r.g  'i'.o  number  of  successful  trials  preceding  each  failure.  The  estimate  of  scstem 


reliability  is  computed  as  the  ratio  of  adjusted  successful  trials  to  adjusted  total  trials. 
Failure  discounting  allows  the  model  to  track,  changing  reliability. 

The  remaining  two  models  are  modifications  of  an  exponential  single  phase 
reliability  estimate  of  the  form  reliability  =  1  -  e^.  One  model  uses  regression  to 
estimate  A.  This  model  is  referred  to  as  the  Exponential  Regression  Model.  The  second 
model  uses  an  unbiased  estimate  of  A  to  produce  a  single  testing  phase  reliability 
estimate.  The  current  reliability  estimate  is  a  weighted  average  of  the  previous  testing 
phase  reliability  estimate  and  the  current  single  phase  reliability  estimate.  The  fraction 
of  the  total  test  trials  to  date  used  m  the  computation  of  each  estimate  becomes  the 
weighting  factor  for  :he  arithemaiic  average.  As  with  the  Maximum  Liklihood  ’.Vi:h 
Discounting  Model,  a  failure  discounting  method  is  applied  to  the  test  data  before 
either  model  is  used  to  compute  a  reliability  estimate.  Both  these  models  have  the 
ability  to  track  changing  reliability  without  the  use  of  failure  discounting;  but.  failure 
discounting  is  employed  '.o  enhance  the  models  performance. 

.A  Mcnte-Cario  simulation  was  uTitten  :o  evaluate  the  performance  of  the 
reliability  growth  models  and  failure  discounting  methods  developed  in  the  study.  This 
was  necessary  because  closed  form  solutions  for  the  statistical  properties  of  the  models 
are  not  mathematically  tractable.  The  simulation  uses  input  parameters  to  generate  a 
reliability  growth  pattern.  Test  data  including  the  number  of  trials  up  to  and  including 
system  failure  and  the  cause  of  system  failure  are  generated  from  the  established 
reliability  growth  pattern.  Each  of  the  three  reliability  growth  are  applied  to  the 
simulated  test  data  resulting  in  an  estimate  of  system  reliability  at  each  testing  phase. 
.After  this  procedure  is  replicated  a  suitable  number  of  times,  estimates  for  the  mean 
and  standard  deviation  of  each  estimate  arc  computed.  These  estimates  are  used  to 
evaluate  the  performance  of  the  reiiabiiity  growth  models.  By  varying  the  reiiabiiity 
growth  pattern  anu  ;iie  failure  discounting  parameters,  the  models  performance  over  a 
wide  range  of  situations  may  be  evaluated.  The  conclusions  reached  from  this  analysis 
are  in^-luded  :r.  tire  fc'ilov.'ing  section. 

B.  CONCLLSIONS 

liven  d'iOugh  the  anaiv^.i'i  o*' the  reliability  growth  models  performed  in  the  study 
IS  not  con'.prenenMve  enough  to  state  conclusions  with  ccrtaints.  several  general 
..onclusior.s  are  apparent. 

I  lie  Llo;.  d  F  iiinre  Discounting  Method  should  not  be  used  with  the  tb.ree 
reliabiiitv  gro’.'.th  models  developed  in  the  study.  With  few  exceptions,  this  method 


caused  the  growth  models  to  overestimate  system  reliability.  This  tendency  to 
overestimate  was  particularly  evident  at  low  reliabilities  when  little  or  no  actual 
reliability  growth  occurred.  The  few  situations  where  the  method  performed  well,  rapid 
reliability  growth,  were  handled  equally  well  by  the  Straight  Percent  Discounting 
Method.  The  Straight  Percent  Discounting  Method's  flexibility  makes  the  preferred  of 
the  two  methods  for  fractionally  discounting  failures. 

The  Weighted  .Average  Reliability  Growth  Model,  in  its  current  form,  should  not 
be  used  to  estimate  system  reliability.  The  model  grossly  overestimated  reliability  for 
most  growth  patterns  tested.  Even  when  no  failure  discounting  was  applied,  the  model 
overestimated  system  reliability.  .Adding  either  failure  discounting  methoa  omy 
increased  the  tendency  to  overestimate. 

Both  the  Maximum  Liklihood  Estimate  With  Discounting  and  the  Exponential 
Regression  Model  exhibited  performance  supenor  to  the  standard,  single  phase  VILE, 
estimate.  But  there  is  no  clear  cut  choice  between  the  Maximum.  Likelihood  Estimate 
With  Failure  Discounting  .Vlodel  and  the  Exponential  Regression  .Model.  Both  models 
accurately  track  a  wide  range  of  reliability  growth  patterns.  The  MLE  With 
Discounting  Model  appears  to  be  the  less  variable  of  the  two  methods  but  is  miuch 
more  sensitive  to  the  choice  of  .failure  discounting  parameters  than  is  the  Exponential 
Regression  Model.  The  variability  of  both  models  decreased  with  increasing  reliability 
and  additional  test  data.  For  this  reason  the  Exponential  Regression  Model  is 
recommended  for  situations  where  four  or  more  testing  phases  are  planned  and  the 
system  possesses  high  reliability  with  near  certainty.  For  other  cases  the  Maximum 
Likelihood  Estimate  With  Failure  Discounting  is  recommended  because  of  its  reduced 
variability. 

The  discrete  reiiability  growth  models  using  failure  discounting  developed  in  the 
study  represent  an  improvement  over  conventional  reliability  estimation  techniques  for 
the  special  case  of  testing  until  a  fixed  number  of  failures  have  been  observed  with 
attr.bute  data.  Howe’.cr.  these  models  can  be  improved.  The  following  section 
contains  recomendations  for  improvements  and  further  stuuy. 

C.  RECOMENDATIONS 

The  following  are  rccom.enJations  for  improvement  to  the  models  introduced  in 
this  study. 

•  Many  more  si.mulaticn  runs  with  varied  reliability  growth  patterns  and  I'ailurc 
discounting  parameters  are  required  before  solid  conclusions  concerning  the 
properties  of  the  reliability  growth  models  can  be  reached. 


The  Monte-Carlo  Simulation  should  be  modified  so  declining  reliability  can  be 
modeled. 

An  alternate  weighting  technique  should  be  developed  for  the  Weighted 
Average  Model  to  correct  its  tendency  to  overestimate  reliability.  In  certain 
situations,  this  model  demonstrated  the  smallest  variability  of  the  three  models 
developed.  The  model  might  prove  to  be  an  excellent  reliability  estimator  if  the 
overestimation  tendency  can  be  corrected. 

An  additional  failure  discounting  method  that  combines  the  lle.xibility  of  the 
Straight  Percent  Discounting  Vlethod  with  the  non-arbitraiy  selection  of  the 
discounting  fraction  from  the  Lloyd  method  might  Drove  useful.  This  could  oe 
accomplished  by  adding  a  discount  interval  to  the  current  Lloya  Discounting 
Method. 

The  possibility  of  using  a  simulation  similar  to  the  one  developed  here  to  aid  in 
the  selection  of  failure  discounting  parameters  should  be  explored.  Using  an 
estimate  of  initial  reliability,  a  reliability  goal  and  the  number  of  planned  trials; 
the  various  choices  for  failure  discounting  parameters  could  be  tested  and  one 
set  chosen  based  on  the  simulation  results. 

Further  analysis  of  the  performance  of  the  reliability  growth  models  with 
varcing  failure  discounting  parameters  is  required  to  develop  "rules  of  thumb" 
for  failure  parameter  selection. 


APPENDIX  A 

DISCRETE  RELIABILITY  GROWTH  MODEL  SIMULATION 
1.  FORTRAN  CODE  FOR  THE  SLMULATION 

•k  X 

*  DISCRETE  RELIABILITY  GROWTH  SIMULATION  « 

X  X 

^  PROGRAMMED  EY  JAMES  E  DRAKE  * 

X  X 

LAST  MODIFIED  19  AUG  1987 


THE  FOLLOWING  EXTERNAL  FILES  ARE  USED  EY  THE  PROGRAM 


INPUT 

THESIS 

RELIAB : 


MLEWD 


MLESP 


REGEST 


:  DATA  AND  PARAMETER  INPUT  FILE  (DEVICE  #  10) 

:  OUTPUT  FILE  CONTAINING  INTERMEDIATE  COMPUTATIONS 
(DEVICE  #  20) 

OUTPUT  FILE  CONTAINING  FINAL  RESULTS  OF  THE  SIMULATION 
(DEVICE  #  30) 

:  OUTPUT  FILE  CONTAINING  EACH  PHASE  ESTIMATE  FOR  EACH 
REPLICATION  OF  THE  WEIGHTED  AVERAGE  ESTIMATE 
(DEVICE  #  40) 

:  OUTPUT  FILE  CONTAINING  MLE  ESTIMATES  USING  DISCOUNTING 
FOR  EACH  PHASE  AND  EACH  REPLICATION 
(DEVICE  #  50) 

:  OUTPUT  FILE  CONTAINING  MLE  ESTIMATE  FOR  EACH  SINGLE  PHASE 
AND  ALL  REPLICATIONS  USING  NO  DISCOUNTING 
(DEVICE  #60) 

:  OUTPUT  FILE  CONTAINING  EACH  PHASE  ESTIMATE  FOR  EACH 
REPLICATION  OF  THE  EXPONENTIAL  REGRESSION  ESTIMATE 
(DEVICE  #  70) 


THE  FOLLOWING  IS  A  LIST  OF  KEY  ARRAYS  USED  IN  THE  SIMULATION 

A  ;  MAIN  WORKING  ARRAY  CONTAINS  PROBABILITY  OF  SUCCESS  FOR 

EACH  FAILURE  CAUSE,  NUMBER  OF  TRIALS  UNTIL  FAILURE  FOR 
EACH  FAILURE  CAUSE  AND  THE  SYSTEM,  CAUSE  OF  FAILURE, 
PHASE  NUMBER,  ADJUSTED  NUMBER -OF  TRIALS  AND  ADJUSTED 
NUMBER  OF  FAILURES 

DIMENSION  (  ( (2*#CAUSES)+7) ,#FAILURES  ) 

NFAPH  :  CONTAINS  THE  NUMBER  OF  FAILURES  IN  EACH  PHASE 
DIMENSION  ( 1 , #?HASES) 

NFCAUS  ;  BINARY  ARRAY  USED  TO  DETERMINE  IF  A  FAILURE  OCCURRED  IN 
A  PHASE 

DIMENSION  (  1,#FAILUR£  CAUSES) 

NTRIAL  :  CONTAINS  THE  NUMBER  OF  TRIALS  SINCE  LAST  FAILURE  OR 
DISCOUNTING  FOR  EACH  FAILURE  CAUSE 
DIMENSION  (  1,#FAILURE  CAUSES  ) 

PHREST  :  RECORDS  THE  PHASE  ESTIMATE  FOR  EACH  ESTIMATOR  WITHIN  A 
SINGLE  REPLICATION 
DIMENSION  (4,#PHASES) 

ROW  1  :  WEIGHTED  AVERAGE  ESTIMATE 
ROW  2  :  MLE  WITH  DISCOUNTING 
ROW  3  :  SINGLE  PHASE  MLE 
ROW  4  :  EXPONENTIAL  REGRESSION  ESTIMATE 
AREL  :  CONTAINS  ACTUAL  SYSTEM  RELIABILITY  IN  EACH  PHASE 
DIMENSION  ( 1 , #?HASES ) 

YJK  :  CONTAINS  YJK  VALUES  UP  TO  1000 
DIMENSION  (1,1000) 

CLTISF  :  CONTAINS  THE  NUMBER  OF  SUCCESS  AND  FAILURES  FOR  EACH 
FAILURE  CAUSE  (USED  WITH  WEIGHTED  AVERAGE  EST.) 
DIMENSION  (3,#FAILURE  CAUSES) 

ROW  1  :  NUMBER  OF  FAILURES 
ROW  2  :  NUMBER  OF  SUCCESSES 
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on  ( i n ( > o o o n n o 


*  ROW  3  :  ADJUSTED  NUMBER  OF  SUCCESSES  * 

*  REG  ;  ARRAY  USED  TO  COMPUTE  THE  EXPONENTIAL  REGRESSION  ESTIMATE  * 

*  DIMENSION  (5,#PHASES)  ^ 

*  ROW  1  :  K  BAR  * 

*  ROW  2  :  Y  BAR  * 

*  ROW  3  :  Y  BAR  FOR  THE  PHASE  * 

*  ROW  4  :  B  HAT 

*  ROW  5  :  A  HAT 

THE  REMAINING  ARRAYS  ARE  USED  TO  COMPUTE  THE  MEAN  AND  VARIANCE  ^ 


-k 

OF  EACH 

ESTIMATE 

AT  EACH  PHASE.  THEY  ALL  HAVE  THE  SAME  DIMENSIONS 

k 

AND  STRUCTURE 

k 

k 

DIMENSI 

ON  (4,tiPHASES) 

k 

k 

ROW  1  : 

RUTINIMG  SUM  OF  ESTIMATES 

k 

k 

ROW  2  : 

RUNNING  SUM  OF  SQUARED  ESTIMJITES 

k 

k 

ROW  3  : 

MEAN  OF  THE  ESTIMATES 

k 

k 

ROW  4  : 

STANDARD  DEVIATION  OF  THE  ESTIMATES 

k 

k 

EST 

VALUES 

FOR  THE  WEIGHTED  AVERAGE  ESTIMATE 

r 

k 

MLEWD 

VALUES 

FOR  THE  MLE  WITH  DISCOLT4TING 

rr 

k 

MLESP 

VALUES 

FOR  THE  SINGLE  PHASE  MLE 

T 

k 

k 

REGEST 

VALUES 

FOR  THE  EXPONENTIAL  REGRESSION  ESTIMATE 

k 

k 

C  DEFINE  AND  DIMENSION  VARIABLES 

PARAl-IETER  (MR=50  ,  MC=200 ) 

INTEGER  REP,CUMSF,DISCPT 
REAL’^4  MIN 

REAL’^3  DSEED,MLESP,MLEWD,EST 

DIMENSION  NFAPH(NR) ,A(NR,NC) ,NFCAUS(NR} ,NTRIAL(NR) ,PHREST(4,NR) ,ES 
CT(4,NR) ,MLEWD(4,NR) ,MLESP(4,NR) ,REGEST(4,NR) ,AREL(NR) , YJK( 1000 ) , CU 
CMSF(3,NR) ,REG(5,NR) 

C  READ  IN  THE  NUMBER  OF  CAUSES  TO  BE  USED  {  NCAUSE  )  AND  THE  NUMBER 
C  OF  PHASES  (  NPHASE  )  IN  THE  TEST 

READ (10,*)  NCAUSE 
READ (10,*)  NPHASE 

CREATE  VARIABLES  FOR  THE  ROW  INDICES  OF  THE  WORKING  MATRIX  (  A  ) 

IPFASE :  PH^iSE 

ISYSPrI  ACTUAL  COMPONENT  RELIABILITY 
INTR;  NUMBER  OF  TRIALS  UP  TO  AIJD  INCLUDING  FAILURE 
IFAILC:  CAUSE  OF  THE  FAILURE 

lADJF:  ADJUSTED  NUMBER  OF  FAILURES  El 

AFTER  DISCOUNTING  HAS  BEEN  APPLIED 
lADJT:  ADJUSTED  NUMBER  OF  TRIALS  AFTER  DISCOUNTING  HAS  BEEN  APPLIED 
lYJK:  YJK  COMPUTED  ON  THE  ADJUSTED  NUMBER  OF  TRIALS 

IPHASE  =  (2*HCAUSE)+1 
ISYSPR  =  IPHASE  +1 
INTR  =  ISYSPR  +  1 
IFAILC  =  INTR  +  1 
lADJF  =  IFAILC  +  1 
-r.DJT  =  I.-.D-c  T  1 
lYJK  =  lADJT  1 


READ  IN  THE  NUTIBER  OF  FAILURES  IN  EACH  PHASE  (  NFAPH(I)  )  AND 
COMPUTE  THE  TOTAL  NUMBER  OF  FAILURES  IN  THE  TEST  ^  NFAIL  ) 

NFAIL  =  0 

DO  10  1=1, NPHASE 

READdO,^)  NFAPH(I) 

NFAIL  =  NFAIL  +  NFAPH(I) 

10  CONTINUE 

C  INPUT  THE  PROBABILITY  OF  SUCCESS  IN  A  SINGLE  TRIAL  FOR  EACH  CAUSE 
C  IN  THE  FIRST  PHASE 
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no  1)  n  n  on  oofTnoonnonno 


DO  20  I=1,NCAUSE 

READ(10,*)  A(I,1) 

20  CONTINUE 

INPUT  THE  REMAINING  VARIABLES  ,  THE  NUMBER  OF  SUCCESSFUL  TRIALS 
BEFORE  A  DISCOUNT  IS  APPLIED  (N) ;  THE  DISCOUNT  FACTOR  (R) ;  THE  SEED 
FOR  THE  RANDOM  NUMBER  GENERATOR,  GGUBFS ,  (DSEED)r  RELIABILITY 
GROWTH  FRACTION  (FRIMP) :  TRIGGER  FOR  PRINTING  INTERIIEDIATE  OUTPUT 
(lOPT) 

TRIGGERS  FOR  SAVING  EACH  ESTIMATE  AT  EACH  PHASE  FOR  EACH  ESTIMATOR 
lOPTl  :  WEIGHTED  AVERAGE  MODEL 
I0PT2  :  MLE  WITH  DISCOUNTING 

Tp.p'T’'?  .  qTMGTr  ’f’’ F 

i0PT4  1  EXPONENTIAL  REGRESSION  MODEL 
DISCOUNTING  OPTION  TRIGGER  ^DISOPT);  LLOYDS  FAILURE  DISCOUNTING 
PARAMETER  (GAMMA) 


RE.AD  ( 

;lo, 

=  ■) 

M 

RE.AD  ^  10, 

T  \ 

READ( 

,10, 

DSEED 

RE.AD  ( 

10, 

•k  \ 

FRIMP 

READ  1 

10, 

k  \ 

NREP 

READ  ( 

10, 

k  \ 

lOPT 

READ  ( 

10, 

ICPTl 

READ  ( 

!io. 

k  \ 

IOPT2 

READ  1 

^10 

k  \ 

/ 

IOPT3 

RE.AD  ^  10 

/ 

lOPT^ 

.RE.AD  (10 

DISC?' 

.RE.AD  (10 

GAIIA 

aTIRE?  =  MRE? 

DSEEDl  =  DSEED 

INITIALIZE  THE  ARRAYS  USED  TO  COMPUTE  THE  MEAN  AND  STANDARD  DEVIATION 
OF  EACH  ESTIMATOR 

DO  30  J=1,MPKASE 
DO  50  1=1,4 

EST(I,J)  =  0.0 
MLEWD{I,J)  =  0.0 
MLESP(I,J)  =  0.0 
REGEST(I,J)  =  0.0 
?HREST(I,J)  =0.0 

30  CONTINUE 

COMPUTE  A.ND  STORE  THE  YJK  VALUES  UP  TO  1000 

YJK(l)  =0.0 
CO  40  1=1,999 

YJK(Irl)  =  YJK(I)  +  l.O/I 
40  GCNTIMUE 

COMPUTE  AND  STO.RE  X  3AR  FOR  THE  EXPONENTIAL  REGRESSION  MODEL 

SUM  =0.0 
DO  50  I  =  1,N?H.ASE 
S’UM  =  SUM  +  I 
REG ^1,1:  =  SUM/ I 
£0  CONTINUE 


iula: 


CM  LOOP 


,;;rei 


FAILURE  CAUSE  VECTOR  (NFCAUS)  AND  (CUMSF) 
INITIAL  SYSTEM  RELIABILITY 


REL  =  1. 

CO  60  I=1,NCAUSE 

NFCAUS(I)  =  0 
REL  =  REL  A  (1,1) 
DO  60  J=l,3 

CUMSF(^J,I)  =  0 

60  CONTINUE 
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C  INITIALIZE  COLUMN  (FAILURE  #  )  COUNTER  FOR  THE  WORKING  ARRAY  (A) 
J  =  1 

C  LOOP  TO  COMPUTE  THE  NUMBER  OF  TRIALS  UP  TO  AND  INCLUDING  FAILURE 
C  AND  THE  CAUSE  OF  FAILURE  FOR  EACH  FAILURE  IN  EACH  PHASE 

DO  130  K=1,NPHASE 

C  SKIP  ACTUAL  COMPONENT  RELIABILITY  COMPUTATION  AFTER  FIRST  RE? 

C  AND  FOR  FIRST  FAILURE 

IF(J.EO.l)  GOTO  75 
IF(REP:GT.:;  GOTO  75 
REL  =  i. 

C  COMPUTE  MEW  ACTUAL  RELIABILITY  FOR  THE  COMPONENT  IN  PHASE  K 
DO  70  I=1,NCAUSE 

C  INCREASE  CAUSE  ?R( SUCCESS)  IF  IT  CAUSED  FAILURE  IN  THE  PREVIOUS 
C  COMPUTE  NEXT  PHASE  RELIABILITY  .\ND  REINITIALIZE  NFCAUS 

IF(MFCAUSfl) .EO.l)  THEN 

A(I,J)  =  A(I,:J-i))  +  ((1.  -  A(I,(J-1)))-FRIMP) 
ELSEIF(NFCAUSvI/ .NE.l)  THEN 
A(I,J)  =  A(I,^--1,) 

ELSE 
END  IF 

REL  =  REL^A^I,J) 

NFCAUS ; I ;  =  0 
"3  CONTINUE 

75  J1  =  1 

TRTCT  =3.0 

C  COMPUTE  THE  NUMBER  OF  TRIALS  UP  TO  AND  INCLUDING  FAILURE  .AND  THE 
C  CAUSE  OF  FAILURE  FOR  EACH  FAILURE  IN  THE  PHASE 

DO  120  L=l,NFAPK(:-<) 

IF(REP.GT.l)  GOTO  90 
IF(Jl.EQ.l)  GOTO  35 
DO  80  I  =  1,NC.AUSE 

A(:,J)  =  A(I,(J-1)) 


80 

CONTINUE 

35 

A(ISYS?R,J)  = 

A( I PHASE , J ) 

90 

MIN  =  7.2E75 

DO  110  I=1,NCAUSE 

C  ASSIGN  #  TRIALS  FOR  C.AUSES  WITH  PR(SUCCESS)  =  0  OR  1 

IF(A(I,J) .GE.l.)  THEN 

A(' (I+NCAUSE)  ,U)  =  7.2E75 
GOTO  100 

ELSEIF(A( I , J  ;  .EQ.O. )  THEN 
A(  ('I^NCAUSE)  ,  J)  =  1. 

GOTO  100 
ELSE 
END  IF 

C  CONVERT  UNIFCR.M  ''0,1)  RANDOM  VARIABLE  TO  GEOMETRIC  (S  TRIALS  UNT 
C  FAILURE  )  FOR  EACH  FAILURE  C.AUSE.  RECORD  THE  MIN  it  TRIALS  FOR  TH 
C  CAUSES  AS  THE  SYSTEM  =  TRIALS  UP  TO  .AiND  INCLUDING  FAILURE  AliD 
C  RECORD  THE  FAILURE  C.AUSE 

A'  ( I*MCAUSE;  .  j;  =  INT^  1  .-raOG(GGUEFS(DSEED)  )/L3G(  A(I ,  J)  ; 
100  IF(A('fI-NCAUSE)  ,  J)  .LE.MIN)  T.HEH  ' 

MIN  =  A( (I+MCAUSE) , J) 

IMIN  =  I 
ELSE 
END  IF 

110  CONTINUE 

A(IFAILC,J)  =  IMIN 
NFCAUS (IMIN)  =  1 

C  COMPUTE  THE  TOTAL  #  OF  TRIALS  FOR  THE  MLE  SINGLE  PHASE  ESTIMATE  AND 
C  INCREMENT  FAILURE  #  COUNTERS 


mu 


A(INTR,J)  =  MIN 
TRTOT  =  TRTOT  +  A(IMTR,J) 
J  =  J  +  1 
J1  =  J1  +  1 
120  CONTINUE 


C  COMPUTE  THE  MLE  ESTIMATE  OF  COMPONENT  RELIABILITY  FOR  THIS  PHASE  AIID 
C  COMPUTE  THE  RUIINIL'G  SUM  OF  ESTIILATES  AND  THE  SUM  OF  ESTIMATES  SQUARE 
C  FOR  COMPUTATION  OF  THE  MEAN  AND  STANDARD  DEVIATION  OF  THE  ESTIMATE 


C 


?HRE5T(3 ,K) 

=  (TRi: 

:t 

- 

NFAPH(K^ )/TR 

TOT 

ML 

,ESP{  1  ,X1 

=  MLESP- 

'  1 

K) 

+  ?HRE3T(3, 

k; 

ML 

FSP{2 ,K) 

=  MLESP ! 

-  o 

K ) 

f  ( PHREST ' 3 

130 

TINUE 

INI 

TI.A 

LI2E  THE 

adjuste: 

)  N 

UMBER  OF  FAILURES 

TO  1  .AND  THE 

NUT! 

HER 

::  d  - .  J  V-  - 

T 

•JRE  OR  DISCO 

unt: 

MIG  ;NTRIALS( 

IN 

PEE 

PARATlSir 

FOR  THE 

»-  T 

C 

CCririT’G  RCLT 

INE 

:G  140  NFAIL 

A( lADJF, J)  =  1. 
140  CONTINUE 


DC  150  I=1,NCAUSE 

NTRIALu)  =  0 
150  COrJTINUE 


DISCOUNTING  ROUTINE  REV 

a;;d  determines  ;;-__the  d 


SENS  ALL  PAST  FAILUPES  AND  CAUSES  TO 
ISIOUNTING  lONDITIONS  HA'.'E  BEEN  MET. 
THE  ADJUSTED  4  OF  TRIALS  AND  YJK 


J  =  0 

DO  300  K=1,N?HJ^SE 
DO  200  L=1,N'FAPH(K) 

J  =  J  +  1 

C  UPDATES  THE  NUMBER  OF  TRIALS  SINCE  FAILURE  OR  DISCOUNTING  FOR  EACH 
C  FAILURE  CAUSE 

ICAUSE  =  INT(A{IFAILC,  J)  +  .  5'. 

DO  160  I=1,NCAUSE 

IF ( ICAUSE. EQ. I)  THEN 
NTRIAL(I)  =  0 
ELSEIF( ICAUSE. NE. I)  THEN 

NTRIAL(I)  =  NTRIAL(I)  +  INT(A( INTR , J ) + . 5 ) 

ELSE 
END  IF 

■  160  CONTINUE 
200  CONTINUE 


C  CHOOSE  DISCOUNTING  METHOD  TO  BE  USED 
IF(DISOPT.NE.2)  GOTO  130 
C  PERFORM  LLOYDS  FAILURE  DISCOUNTING  METHOD 

DO  170  1=1, J 

II  =  INT(A(IFAILC,I)+.5) 
IF(NTRIAL(I1 ) .EQ.O)  THEN 
A{IADJF,I)  =1.0 
GOTO  170 


A(IADJF,I)  =  1.0  -  ((l.-GAMA)**(1.0/NTRIAL(Ii))) 
170  CONTINUE 
GOTO  210 

C  PERFOR.MS  STRAIGHT  PERCENT  FAILURE  DISCOUNTING  AND 
C  COMPUTES  THE  ADJUSTED  #  OF  FAILURES 

130  DO  190' 1=1, J 

II  =  INT(A(IFAILC,I)+.5) 

IF(NTRIAL(I1) .EQ.O)  THEN 
A(IADJF,I)  =  1. 

ELSEIF(NTRIAL(I1) ,GE.N)  THEN 
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l‘l  I  > 


A(IADJF,I)  =  A(IADJF,I)*((1.-R)**(NTRIAL(I1)/N) ) 

ELSE 
END  IF 

190  CONTINUE 

C  ADJUSTS  THE  #  TRIALS  SINCE  FAILURE  OR  DISCOUNTING  FOR  THOSE  CAUSES 
C  THAT  HAVE  MET  OR  SURPASSED  THE  DISCOUNTING  THRESHOLD 
C  FOR  THE  STRAIGHT  PERCENT  DISCOUNTING  METHOD 

DO  205  I=1,NCAU3E 

IF(NTRIAL(I) .GE.N)  NTRIAL(I)  =  MOD (NTRIAL ( I ) , M ) 

205  CONTINUE 

210  TADJT  =0.0 
TYJK  =0.0 
TPYJK  =0.0 
K1  =  0 

DO  215  12=1,3 
DO  215  I=1,MCAUSE 
C'JMSF(I2,I)  =  0 
215  CONTINUE 

C  COMPUTES  THE  ADJUSTED  #  OF  TRIALS  FROM  THE  ADJUSTED  #  OF  FAILURES 
C  A:ro  COMPUTES  THE  SUM  OF  THE  ADJUSTED  #  OF  TRIALS  FOR  ESTIMATE  COMP. 

PREL  =0.0 
LTRIAL  =  0 

DO  240  :=i,j 

A(IADJT,I)  =  A(:NTR,I)/A(IADJF,I) 

TADJT  =  TADJT  +  AUADJT,!) 

C  COMPUTE  YJK  FROM  THE  ADJUSTED  S  OF  TRIALS  AND  STORE  THE  SUM  FOR 
C  ESTIMATE  COMPUTATION,  USE  ARRAY  FOR  #  TRIALS  <  1000  AND  APPROX.  FOR 
C  VALUES  >  1000 

N1  =  NINT^AdADJT,!)) 

IF (Ml. LE. 1000)  THEN 

A(IYJK,I)  =  YJK(Nl) 

ELSEIFa-Jl.GT.lOOO''  THEN 

A(IYJK,I)  =  (A(IADJT,I)**.10995)*3.55445 

ELSE 
END  IF 

C  DETERMINE  IF  A  PHASE  BOUTJDARY  HAS  BEEN  REACHED  TO  BEGIN  ESTIMATE 
C  COMPUTATION 

IFd.EQ.l)  GOTO  225 

IF''A(I?HASE  ,  I ,  ..NE.A(  IPHJ^SE  ,  (I-l)  ) )  THEN 
0  COMPUTE  THE  'WEIGHTED  AVEPJ^GE  ESTIMATE 

MAX  =  0 
K1  =  K1  *  1 

C  DETERMINE  THE  FAILURE  CAUSE  WITH  THE  LARGEST  OF  FAILURES 

DO  220  I1=1,NCAUSE 

I F  I"  CUMS  F  ;  1  ,  1 1  )  .  GT  .  :iAX )  THEN 
MAX  =  CUMSF( 1 .11/ 

:::l  =  :: 


C  COMPUTE  YJK  VALUE  FOR  THE  CURRENT  PHASE  ESTIMATE 

IFdUMSF(  1  ,  I  COL  )  .LE  .  ICCO  ;  THEN 
AHATL  =  YJK' CUMSF ■ 1 , ICOL n 
ELSEIF.'CUMSF' :  I  COL  .  GT  .  1  GOO  )  T.HEN 
X  =  CUMSFd  ,  I  JC’L  , 

AHATL  =  'X-*.  10955  d3. 55445 
ELSE 
END  IF 

IX  =  CUMSF( 1 , ICOL)  +  CUMSF(3,ICOL) 
IF(IX.LE.IOOO)  THEN 


LTRIAL  =  CUMSF(1,IC0L)  +  CUMSF(3 , ICOL) 
REG(2,K1)  =  TYJK/(J) 

(3,K1  =  - 


REG 


TPYJK/NFAPH(K1) 

PHREST(1,K)  =  PREL 

COMPUTE  THE  HLE  ESTIMATE  OF  PHASE  RELIABILITY  USING  DISCOUNTING 
?KREST(2,K)  =  (TADJT  -  J;/TADJT 


:0M?UT; 


THE  z:-:pcme::t:al  regression  estimj^te  begintiimg  with  b  hat 

SUM  =0.0 


.3U 


SUMS  =1.0 

IF  (K.EO.l 

)  GOTO  252 

DO  250  !  = 

1  ,K 

SUM 

=  S  L’M  (  ( I  -  i 

SUMS 

=  SUMS  r 

CONTINUE 

?,eg;4.m,  = 

SUM/ SUMS 

MPUTE  A  HAT 

REG.S.K;  = 

reg;2,k)  - 

MPUTE  AND  STCRE 

THE  EMPCMENT 

'2; 


?HP.E5T^4.K)  =  l.Q  -  EM?  (  -  ^  REG  v  5  ,  K )  -  •  REG  U  ,  K)  *K  ;  :  ) 
IF  ?HRE5T;4,M',  .LT.  :  PKREST  <  4  ,  M ;  =0 . 0 


-  rv-D  -  _3; 


iF'RHREST'  -i.R/  .  IT  .  I 


,K)=o. : 


:  STORE  THE  ROTir.'I.MG  SOM  IF  THE  ESTIMATES  FOR  THE  CURRENT  PHASE  AND 
:  RUNi;iNG  SUM  IF  THE  ESTLIATES  SQUARED  FOR  COMPUTATION  OF  THE  MEAN 

c  st.ai;:ard  deviation  of  each  est!jlate  for  each  reliability  growth 

0  MODEL 


255 


EST(l.K)  =  ESTC.K)  +  PHRESTd.K^ 
EST:2,K,  =  EOT;:  K.  -  >?HREST;I  K)*^2) 


MLEWC ( I 

,.M/  =  MLE 

ViC  ’ 

)  + 

PHREST! 2,K) 

MLEWD  <'  2 

K'  =  MLE 

WT  ■ 

!k 

)  + 

!?HREST^2  ,K^**2) 

REGEST ; 

i  u<)  =”re 

GE5 

(1 

,K) 

PHREST  f  4,  K) 

REGEST',' 

=  RE 

GES 

T- 

'  2 

,K) 

vPHREST;  4,K)*^ 

THE  ACT', 

!nL  PHASE 

REII 

:abil: 

1  X 

.AREL(K) 

-  n  i  0  i  O 

?F, 

) 

1040  FORMAT (4X, 'FAIL  #',3X,'FAIL  CAUSE', 3X,'#  TRIALS 3X AD J  #  FAIL', 3 
CX,'ADJ  #  TRIALS' ,7X, 'YJK' ) 

DO  270  1=1, J 

WRITE(20,1050)I,A(IFAILC,I),A(INTR,I) ,A(IADJF,I) ,A(IADJT,I)  ,A(IYJK 
C,I) 

1050  FORMAT(4X,I3,8X,F3.0,7X,Fa.0,4X,F8.6,4X,F12.0,3X,Fll .4) 

270  CONTINUE 

WRITE(20,1060) 

1060  FCR1-IAT(  ‘  '  .ill) 

300  CONTINUE 

C  PRINT  EACH  OF  THE  3  ESTIMATES  TO  THEIR  APPROPRIATE  OUTPUT  FILE 
C  IF  REQUESTED 

IF(IOPTl.NE.l)  GOTO  401 

400  WRITE (40, 2000^  ( PHREST ( 1 , I ) ,  I=1,NPHASE) 

401  I?(IOPT2.NE.l')  GOTO  402 

WRITE (50. 2000)  ( PHREST . 2 , I ) ,  I=1,MPHASE( 

402  IF(:OPT3.NE.l)  GOTO  402 

write: 50, 2000  )  '  PHREST '2, 1),  i=i,:ifhase; 

403  if;iopt4.;:e..  )  goto  £oo 

•WRITE  (70, 2000)  ( PHREST 4 , 1 )  ,  I  =  1,NPHASE) 

2000  FGRMAT('  ' , 20 (F7 . 6 : IX) ; 

500  CONTINUE 

C  UPON  COMPLETION  OF  ALL  REPETITIONS,  COMPUTE  THE  MEAN  AND  STANDARD 
C  DEVIATION  OF  EACH  ESTIMATE  FOR  EACH  PHASE  SKIPPING  COMPUTATIONS  IF 
C  ONLY  ONE  REPETITION  IS  REQUIRED 

IF  (NP.EP.LE.l/  GOTO  501 

DO  500  I=I.MPHASE 
EST(3,I)  =  EST(  1 . 1  )/X'NRE? 

MLEWD  (3,1)  =  MLEWD  <1,1;  ,T<NREP 
HLESP(3,I)  =  MLESPd  ,  I)/XNRE? 

REGEST(3,I)  =  REGEST(1,I)/>DJREP 

EST(4,I)  =  SQRT((EST(2,I)-(:<NREP’''(EST(3,I)**2)))/(XNREP*r/) 

MLEWD (4, 1 ')  =  SQRT(  (MLE’WD(2,I)-(XNREP’'(MLE'WD(  3  , 1  )*'^2  )  )  ) /  :  FD'IREP- 1 )  ) 
MLESP(4,I)  =  SQRTl  (MLESP(2,I)-(XI'IREP*(;MLES?(3,I)**2)  )/(:-LNREP-l  )  ; 
REGEST  (4,1)  =SQRT  (  ( REGEST  (2,1)-  ( XJRE?’'  ( REGEST  ( 3 , 1 )  **2 ) ) )  /  ( XNREP- 1  ,■ ) 
600  CONTINUE 


C  PRINT  THE  FINAL  OUTPUT  TABLE  TO  A  FILE 
601  WRITE(30 , 3000) 

3000  FCRMJi.Ti  'O'  ,T47  'DISCRETE  RELIABILITY  GROWTH  SIMULATION') 
WRITE(30,3010) 

3010  FORMAT( ‘ ,T54, 'MODEL  PARAMETER  SUMMARY') 

WRITE  30,3020)  NCAUSE 

3020  FCRNAT( ' 0 '  ,T47  ,  'NUMBER  OF  POSSIBLE  FAILURE  CAUSES  ',14) 
•WRITE  (  20  ,  202  0  ' 

2030  format:  0 123 ,  cause  :rjMBER  ,T54,  ■  single  tri.rl  fr(  success 
CP -HAS  E  1  '  ) 

DC  20  50  .M=:,  NCAUSE 
’WRITE  (  20 , 2040  >  M  A'M,  1  ) 

3040  format: '  ,T43,I2,T'9,F3.6) 

3050  CONTINUE 


:06: 


WRITE, 20, 305D)  FRINP 


UJSE  RELIABILITY  IMPROVES  .RFTER 


..I  ..“.c. 


q  ’MT-* 


WRITE  20,2020:  NFHACE 
'060  *  '  “  .  .  4  3  ,  '  NU,2c E R 

'WRITE  (3  2,3  2  02' 

JMA7 
C  ?  HA  5  E  1 

DO  3112  M=: .NFHASE 
WRITE ( 30 , 3  *  :  2  ;  M , NF APH; M ) 

iloo  fcr::aT'  '  •  ,  T43  ,  id  ,  i~3  ,  id  , 

'WFITE^!: , 2120)  NFAIL 

FORMAT',  0  '  ,T51  ,  'TOTAL  M’JMEER  OF  FAILURES  ',14, 
I  F  '  D  I  SORT  .  E'O  .  0  )  O'"  TO  316'0 
WRITE  (  30  ,  3i!C' 


1 1^ 
^  J 


3130  FORMATC ,T38, 'DISCOUNTING  PERFORMED  USING  THE  CONSTANT  FRACTION 
CMETHOD ' ) 

WRITE(30,3140)  R 

3140  FORMATS 'O' ,T44, 'FRACTION  EACH  FAILURE  IS  DISCOUNTED  ',F8.6) 
WRITE(30,3150)  N 

3150  F0RHAT('  ' ,T33, 'NUMBER  OF  TRIALS  AFTER  A  FAILURE  BEFORE  A  DISCOUNT 
C  IS  APPLIED  ' ,14) 

GO  TO  3190 
3160  WRITE (30, 3 170^ 

3170  F0R:IAT( ,T44,  ‘DISCOUNTING  PERFORI’ED  USING  THE  LLOYD  METHOD') 
WRITE (30, 3180)  GAMA 

3130  FORMAT! ' C ' ,T39, 'PERCENT  C.I.  (  USED  AS  DISCOUNT  FRACTION  )  ' , F8 . 6 
C) 

3190  WRITE(30,32G0 )  DSEEDl 

3200  F0R1'!AT( '-'  ,T46, 'RANDOM  NUMBER  SEED  USED  ',F15.2) 

WRITE (30, 3210)  NREP 

3210  FOPJ-!AT( 'O'  ,T37,  'NUMBER  OF  REPETITIONS  OF  THE  SIMULATION  PERFORMED 
C  ,17) 

WRITEdO  ,3220) 

3220  FORMAT!  ,  T6i  ESTIMJ^TOR  :  '  ) 

WRITE ( 30 , 3230 ) 

3230  FORMAT! 'O' ,T43  'SINGLE  PHASE  MLE  WITHOUT  DISCOUNTING') 

WRITE (30, 3240) 

3240  format! ,T50, 'MEAN' ,T33 , 'ESTIMATE' ,T109, ' 95  %') 

WRITE(30,3250) 

3250  FORMAT!'  ',T12. 'PHASE  MUT^BER ', T29 ACTUAL  RELIABILITY T52 PREDIC 
CTED  RELIABILITY'  ,173,  ■ STANDARD  DEVIATION'  , TlOl ,  CONFIDENCE  INTERV 

COMPUTE  2.1.  FOR  SINGLE  ?HA5E  MLE 

DO  3270  M=1,MPHASE 

Cl  =  !1.96^MLESP(4,M))/SQRT!XNREP) 

CIU  =  MLESP!3,M)  +  Cl 
CIL  =  MLES?(3,M)  -  Cl 

WRITE!30,3260)  M,AREL(H) ,MLESP(3,M) ,MLESP(4,M) , CIL, CIU 
3260  FCFJ-!AT( '0'  ,T17,I2,T34,F3.6,T53,F3.6,T32,F9.6,T99,  '  !  ',F3.6,'  ,  ',F 
C8.6, '  ) ' ) 

3270  CONTINUE 

WRITE!30,3220) 

WRITE (30, 3230) 

3280  FORMAT! 'O' ,T42, 'MAM  LIKELIHOOD  ESTIMATE  USING  DISCOUNTED  FAILURES' 
C; 

WRITE(30,3240) 

WRITE(30,32S0) 

COMPUTE  C.I.  FOR  MLE  WITH  DISCOUNTING 

DO  3290  M=1,MFHASE 

21  =  1.96'MLEWD'4,M  SQRT'XNRE?' 

I I  j  =  M  L  :i  W  D  '  i  ,  M  )  ~  II 

III  =  MLEWD  3  m:  -  :i 

WRITE:  30  SZSI';  M.AREL.M)  ,MLEWD(3,M)  ,MLEWD(4,M)  , CIL,  CIU 
3290  :.;:;ti;iuf 

33::  r'FMAT  did  weiihtz:  average  estimate  usi:jg  failure  dis:ou:;t:m 


I 


I 


I. 


WRITE(30,3240) 

WRITE(30,3250) 

C  COMPUTE  C.I.  FOR  EXPONENTIAL  REGRESSION  ESTIMATES 


DO  3330  M=1,NPHASE 


CIU  =  REGEST(3 

,H)  +  Cl 

III  =  REGEST>'3 

,M)  -  11 

WRITE (30, 3260) 

M,AREL.M)  ,: 

330  GOMTINUE 

WRITE! 30,3340  ' 

340  FOF.MAK'l  ,T50, 

'RECAFIT’JLA' 

'■/■RITE  (30,3330  , 

350  FGRGIATl  '  -  '  ,T2  ,  ' 

PHASE  Til  , 

C  T63 , ' EST '  T"3 

.'MEAN'  T33 

wRirE^2o,3:’ia; 

3360  rCRHAT''  Til,  RELI.-.E  '  .  T23  .  '  WGT  '  ,  T33  .  '  5TT  ‘  .  T5  3  ,  '  MLE  '  ,  T6  3  , 
T"  PHASE  T33 .  ST3  1113,  ?.EG  T113,  STD 
WRITE  ,  3  J  ,  33^  j 

3370  ?OR!LAT('  '"TESi'AVG'  T35 , ' DEVTATICN '  , T53  ,  ' W/T '  ,T60,'TEV:aT 
C,;MLE '^^TSS^OEVIATIOM'  ,T103,  ‘  EST  '  ,T110,  ‘DEVIATION'  ; 

'WR  ITEi^30,33"'':' 

3375  FORMAT  '  ,  T23  .  '  EST  ' 

DC  650  I  =  l,:;rHASE 

WRITE  '  30  3  350  '  I  AREL>I  ,EST'3  I;.EST''^  I'  MLEWD  '  3  , 1  '  MI 
:”LES?.3  1  MIES?  4.1  ?.EGEST;3,1  ,REGE5T'-  I' 

3330  30RM:-.T_^;  _T4_  13,T11_F-_.o^T:6,F;;.3,T36,F~.o,T51  F:'.5,T61  F' 
lF~.6,T36,F”.6,TlC.,F“.6,Tlll,F7.i' 

650  ODNTINUE 


, 'MEAN' 


IC!r  ,T''3 


SAMPLE  SIMULATION  INPUT  FILE 


5 

10 

1 

1 

1 

1 

I 

I 

1 

1 

I 

1 

.8 

.8 

.8 


50 


624“12.0 

.‘55 


0 

0 

!■) 


NUMBER  OF  FAILURE  CAUSES 
NUMBER  OF  PHASES  (  NPHASE  ) 

NUMBER  OF  FAILURES  IN  PHASE  1 

NUMBER  OF  FAILURES  IN  PHASE  2 

NUMBER  OF  FAILURES  IN  PHASE  3 

NUMBER  OF  FAILURES  IN  PHASE  4 

NUMBER  OF  FAILURES  IN  PHASE  5 

NUMBER  OF  FAILURES  IN  PHASE  6 

NUMBER  OF  FAILURES  IN  PHASE  7 

NUMBER  OF  FAILURES  IN  PHASE  8 

NUMBER  OF  FAILURES  IN  PHASE  9 

NUMBER  OF  FAILURES  IN  PHASE  10 

I  -  PROB.  OF  OCCURRENCE  FOR  CAUSE  1  IN  PHASE  1 

1  -  PROB.  OF  OCCURRENCE  FOR  CAUSE  2  IN  PHASE  1 

I  -  PROB.  OF  OCCURRENCE  FOR  CAUSE  3  IN  PHASE  1 

1  -  PROB.  OF  OCCURRENCE  FOR  CAUSE  4  IN  PHASE  I 

I  -  PROB.  OF  OCCURRENCE  FOR  CAUSE  5  IN  PHASE  1 

DISCOUNT  interval  FOR  STRAIGHT  PERCENT  DISCOUNTING 

DISCOUNT  FR^ACTION  FOR  STIUAIGHT  PERCENT  DISCOUNTING 

R. ANDOM  NU  MBER  SEED 
RELIABILITY  GROWTH  FIUACTION 

NU.M3ER  OF  DESIRED  REPETITIONS  FOR  THE  SI.MULATION 
INTERMEDIA  ri:  OUTPUT  OPTION  (  LINT.  OUTPUT;  NO  OUTPUTi 
SAVE  ALL  W  LIGHTED  AVER.AGE  ESTI.MATES  i  1;  YES;  0:  NO  ) 

SAVE  ALL  MLE  W  DISCOUNTING  ESTIMATES  (I;  YES;  0:  NO  ) 

S. AM  \I.L  ml:  SINGI.L  PH.VSr  I.SBIM.MFS  li;  8YS.  >:  Nf) 
v\'  i’  \L;.  R!.^U^;.SSIO\  LSEIM.MES  l:^LS.'l:\o 

DiS(  Ol  N  ITu;  oi’l  K.)\  ,  :  SI  RAlGt'  r  ,  2:  LLoYD  MLI  IDD. 
GAMMA  [OR  1  1  OSD  D1S<01  \II\(j  MI  IliOi.) 


3.  SAMPLE  SIMULATION  OUTPUT  FILE 


DISCRETE  RELIAEILITY  SROWTH'  SINULATIOM 


NOOEL  PARAMETER  SUMMARY 
NUMBER  OR  POSSIBLE  FAILURE  CAUSES 


CAUSE  NUMBER 
1 
2 
3 

<t 

S 


SIN6LE  TRIAL  PR(  SUCCESS  ]  FOR  PHASE  I 
0.800000 
o.aooooo 
0.800000 
0.700000 
.  0.700000 


FRACTION  CAUSE  RELIABILITY  IMPROVES  AFTER  FAILURE  0. 750000 


NUMBER  OF  PHASES  IN  THE  SIMULATION  10 

PHASE  NUMBER  NUMBER  OF  FAILURES  IN  THE  FIRST  PHASE 

1  1 

2  1 

3  1 

A  1 

S  I 

i  1 

7  I 

8  1 

»  I 

10  1 

TOTAL  NUMBER  OF  FAILURES  10 


OISCOUNTINO  performed  using  the  constant  fraction  METHOD 
fraction  each  failure  is  discounted  0.750000 

NUMBER  OF  TRIALS  AFTER  A  FAILURE  BEFORE  A  DISCOUNT  IS  APPLIED  50 


RANDOM  NUMBER  SEED  USED 


t2A712.aO 


number  of  repetitions  of  the  SIMULATION  PERFORMED 


A 


RtLIABILlTY 


APPENDIX  B 

GRAPHICAL  RESULTS  OF  THE  RELIABILITY  GROWTH  MODEL  SIMULATION 
I.  MEAN  ESTIMATES  FOR  EACH  ESTIMATOR  WITH  CONSTANT  PARAMETERS 


STD  DISCOUNTING.  N  =  5  ,  F  =  .25 

— i —  actual  reliability 

WEIGHTED  average  ESTIMATE 


— I —  mlE  aith  discounting 
— single  =HASE  MLE 
— RI —  EXPONENTIAL  REGRESSION  EST. 


phase 


STD  DISCOUNTING.  N  =  5  ;  F  -  .50 


- - —  ACTUAL  RELIABILITY 

— . —  WEIGHTED  average  ESTIMATE 

- y...  mLE  with  discounting 

— >« —  single  phase  MLE 
— V--  EXPONENTIAL  REGRESSION  EST 


i  6 

phase 


3 


STD  DISCOUNTING.  N  =  10  ;  F  =  .25 


— i —  actual  reliability 
.........  weighted  average  estimate 

Ml  E  with  mSCOUNflMG 
SINGLE  phase  MLE 

—  9—  EXPONEuriAL  REGRESSION  EST. 


J. 

6 


8 


+ 


phase 


B  0 


PHASt 


LLOYD  DISCOUNTING;  GAMMA 


-  ACTUAL  ReUABILlTY 

WEIGH reo  average  estimate 

-  MLE  with  discounting 

-  SiMGiE  PHASE  MlE 

-  EXPCMEflTlAU  REGRESSION  EST. 


o 


LLOVD  DlSCOUNriNG.  G^MWA 


999 


n  I 

_  f- 
^  1 


— ■ —  Ac:i;AL  RELiABiLirY 

WEiCurED  avERaCE  ESriuAfE 
MLE  AIIH  DiSCCUNriNC 
SiNC;  E  “^'ASE  ui  E 

-  -9  -  ExPE.'.rMT  AL  RE'iRElS.CN  EST 


NO  DISCOUNTING  APPLIED 


■*—  aC^L;AU  ^ELiAftll.r/ 

-viRAcr  r^riMATF 

E  i 

-(£  -'Z'Ji  CN  £3 


LLOYD  DISCOUNTING.  GAWMA 


f)D-fll8C  38a 


UNCLHSSIFIED 


DISCRETE  RELIRBILITV  GROWTH  MODELS  USING  FRILURE 
DISCOUNTINOCU)  NRVRL  POSTQRRDURTE  SCHOOL  HONTEREV  CR 
J  E  DRRKE  SEP  87 


STD  DISCOUNTING,  N  =  10  ; 


RELIABILITY 


V 


STD  DISCOUNTING.  N  -=  20  ; 


— —  ACTUAL  RELIABILITY 

—  WEIGH  TED  AVERAGE  ESTIMATE 

- 1 - MLE  WITH  discounting 

SINGLE  PHASE  MLE 

—9—  EXPONENTIAL  REGRESSION  EST. 


6 

a 

*Q 

phase 

STO  DISCOUNTING.  N  =  20  ; 

— »—  actual  reliability 

wElCHfED  average  estimate 
— -I —  MlE  with  DISCOUNTING 
— SINGLE  Phase  mlE 
—  V---  exPONENIlAL  REGRESSION  EST 


STD  DISCOUNTING.  N  »  20  ;  F  =  .75 


— —  ACTUAL  RELIABIUTY 

WEIGHTEO  AVERAGE  ESTIUATE 
-H —  MLE  WITH  DISCOUNTING 
~M—  SINGLE  PHASE  MIE 
-9—  exponential  regression  est. 


J _ ; _ 1 _ I  I 

4  6  3 

phase 


LLOYD  DISCOUNTING;  GAMMA  =  .9 


WEIGHTED  AVERAGE  ESTIMAIE 
«■  — I—  mlE  wiih  Discounting 

Single  phase  mi  e 

—V—  exponential  regression  est. 

J _ I _ I _ I _ 1 _ \ _ L 

4  6  8  HI 

PHASE 


RELIAElLiriT 


LLOYD  DISCOUNTING;  GAMMA  »  .99 


J _ I _ I _ I _ L 

A  6  a 

PHASE 


LLOYD  DISCOUNTING;  GAMMA  =  .999 


o 


-1 - 1 _ 1 _ I _ L 

“  6  8 
phase 


99 


reliability 


■t  < 


RELIABILITY 


NO  DISCOUNTING  APPLIED 


9- 


••-'  /  / 
/  ' 

^  /  / 

y  / 


- - —  ACTUAL  .'TELIA81UTY 

WEIGHTED  AVERAGE  ESTIMATE 

- 1 - MLE  WITH  DISCOUNTING 

— -K —  SINGLE  PHASE  MLE 

—9—  exponential  regression  EST. 


2 

4. 

6 

8 

10  1 

phase 

STD  DISCOUNTING,  N  =  20  ;  F  =  .25 


/  /  r- 

/  /  j 

-  -i'  / 


,7 


y  / 

' ' 

y 


- -  ACTUAL  reliability 

.  .........  WEIGHTED  average  estimate 

— H MIE  WITH  DISCOUNTING 

Single  phase  mie 

—9—  Exponential  regression  est. 


1  6 

8 

10 

PHASE 

>yv 


RELIABILITY 


LLOYD  DISCOUNTING;  GAMMA  =  .999 


/  - - —  ACTUAL  SeLfAfllLlTY 

,  '  -  wf.iGHreo  average  estimate 

' //  . - 1 —  mle  with  discounting 

'/  — M —  SINGLE  phase  mle 

— 9 —  EXPONET4TIAL  REGRESSION  EST 


G _ I _ : _ I _ L 

A  6  a 


PHASE 


EXPONENTIAL  REGRESSION  ESTIMATE 


— . —  actual  reliability 
.........  NO  discounting 

— i —  STO  DISC.  N  -  5  .  F  -  25 

— SID  DISC.  N  »  5  .  F  -  .50 

—  9 —  STD  DISC;  N  «  5  .  F  »  .75 


MLE  WITH  DISCOUNTING 


ML£  WITH  DISCOUNTING 


ACrUAL  RfLIABlLlIY 
NO  DISCOUNTING 
sro  DISC:  N  =  20  .  F  =■  50 

sro  DISC,  N  -  20  .  F  =  .75 

SIO  DISC.  N  =  t  .  F  =  .10 


RELIABILITY 


111 


EXPONENTIAL  REGRESSION  ESTIMATE 


-■  actual  SELIABIUTY 
...... —  NO  discounting 

— STD  DISC.  N  -  20  ,  «■  -  25 

— « —  STO  DISC;  N  -  20  ,  F  -  50 

STO  DISC.  N  -  20  .  F  »  75 


2 _ I _ I _ _ _ L 

4  6  8 

phase 


EXPONENTIAL  REGRESSION  ESTIMATE 


- - —  actual  REUABIUTY 

- .....  NO  OISCOIINTING 

— K-  '.lOyo  oiscoumtimc.  gamma  »  i 

— —  lLOYO  OISCGUNTING;  ''.AMMA  -  09 
— LLOYO  OISCOUNIING;  OAMMA  -  399 


NEtiAtiLirr 


KLl  .AtJII  il  r  E5T 


ESTIMATOR  VARIABILITY  (  MULTIPLE  BOX  PLOTS 


MLE  W/DISCO'JNTING 

STD  discounting  N  =  10.  R=  25 


2  4  5  8  to 

PHASt  NUW9SP 


WLE  W/DISCOUNTING 
LLOYD  discounting  GAMWA  = 


MLE  W/DISCOUNTING 

STD  DISCOUNTING  N=50,  R  =  .75 


«l  I  lAUlllf  l 


EXPONENTIAL  REGRESSION  E5T. 
STD  discounting  N=20.  R=  25 


phase  number 


EXPCNEN’’’'''.  REGRESSION  EST. 
LLCTO  C'SCC'JNTiNG  GAU^VA  =.999 


exponential  regression  est. 

LLOYD  Discounting  c-awma  =.999 


I 


NUMBEP 


MLE  W/DtSCOUNTING 

STD  OISCOUNflNG  N  =  50.  R=  75 


f  f  f  / 

'  _ x  n  .L 


PHASE  NUMBER 


^ . ^ 


EXPONENTIAL  REGRESSION  EST. 

LLOYD  discounting  GAMMA  =  8 


6  a 

PUASE  NUMSE*? 
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